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How to Use This Guidebook 


This guidebook is divided into four sections and a set of appendices: 

• Background Information 

• Data Team Logistics 

• Data Inquiry Toolkit 

• Sample Data 

• Appendices 

Each section builds on the information in the prior section. Users should first review the guide in its entirety to 
become comfortable with the content and ready to participate as a member of a data team. The Data Inquiry Toolkit 
and Sample Data sections will walk the user through each step of the data analysis and decision-making process. 

The Background Information section provides an overview of the research around school improvement efforts 
and data processes in education. This section offers general background information that provides context prior to 
beginning data inquiry cycle efforts. 

The Data Team Logistics section helps the user identify and build a data team. It provides a set of tools to assist a 
data facilitator or administrator organize the structures for data inquiry. 

The Data Inquiry Toolkit provides the tools needed to analyze any set of data to solve a problem or investigate a 
question. 

The Sample Data section demonstrates how to use the Toolkit by describing a full process using a common school 
improvement question and related data. 

The guidebook also includes an Appendix containing a glossary, examples of visual data displays, and a bank of 
common questions to assist teams as they begin their data analysis efforts. 

The supplementary Facilitator’s Guide, available at the Nebraska Department of Education’s (NDE) website, is 
designed to assist a data team facilitator in successfully leading a team through all of the stages of the data analysis 
and decision-making process. 

Access the NDE website at www.education.ne.gov/APAC/SchoolImprovement.html. 
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The Six Stages of the Cycle of Data Inquiry 
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Background Information 


Introduction 


Schools and school districts are data-rich environments, with heaps of data on student demographics, achievement, 
outcomes, and perceptions. With so much data available, it can be easy to get caught up in the sheer enormity 
of the data sets. A cohesive data analysis and decision-making process can help schools, districts, and additional 


stakeholders to 

• ask good questions, 

• sort through the available data, 

• identify which figures are needed to answer the questions at hand, 

• analyze the data, and, ultimately, 

• use the information gained from the analysis to make good decisions and 
carry out meaningful action for school improvement. 

Data can provide policymakers with objective information needed to make 
decisions about their education systems, provide states with information about 
program effectiveness, and provide schools and districts with information about 
student learning to influence instruction, programming, and teacher professional 
development. 

With recent changes in accountability expectations of educational institutions 
and an increased awareness of the power of data, both schools and districts are 
expected to be data-driven organizations, making decisions and taking action 
based on the facts presented to them through data. This is a considerable charge, 
but one that can lead to powerful improvements in student achievement. 

The goal of this guidebook is to build capacity for data use at the school, district, 
and state levels. Ideally, data users will train their colleagues on the data inquiry 
process, building an entire system of educators who are comfortable with and 
skilled at using data to take action for school improvement. 

Why Have a Data Process? 

It is important to have a process for any activity, to ensure high-quality outcomes 
and seamless transitions. In having a clearly-described, comprehensive process, 
data teams are able to efficiently and effectively work through their questions for 
inquiry, data collection, data analysis, and action tasks. 

The purpose of the data process is to use data to answer important questions in 
order to drive positive change to improve student achievement. Good decisions 


In an elementary school in one 
medium-sized district, the teachers 
and leaders were concerned about 
students' scores and skills in reading. 
They assumed that, based on 
anecdotes and personal experiences, 
the issue lay in a lack of opportunities 
for reading outside of school. The 
staff implemented a costly program 
in which they provided students with 
books to take home and read, with 
associated homework assignments. 
The team expected that, after the 
program had been in place for some 
time, students' skills and scores would 
improve. 

After having expended considerable 
resources, the scores remained flat 
one year later. Confused by this, 
the principal convened a team of 
reading teachers to investigate the 
issue. The team began a process of 
systematically collecting and analyzing 
the data on student reading skills and 
achievement. They found that the 
data indicated that the problem might 
lay in a lack of foundational reading 
skills. After implementing a targeted, 
data-based program of intensive 
skill building in the lower grades, 
the school experienced marked 
improvement in the following year's 
NeSA-R tests. 
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and meaningful actions rarely emerge from assumptions and gut reactions. The data process ensures that schools and 
districts use their resources wisely to take effective action for change. 

The data process guides teams and practitioners toward the root causes of a challenge, allowing them to take targeted, 
effective action to improve the situation without wasting time or other resources on potentially ineffective solutions. It 
helps educators to maximize the impact of their efforts. 

A cohesive data process also enables teams to hold consistent expectations of the actions that can be taken to move 
toward a data-based school or district culture. Over time and with repeated applications, teachers, administrators, and 
other education personnel will share a common language around data use. 

What Makes a Data Process Effective? 

An effective data process is structured and systematic, leading to informed decision making and action. To effectively 
use data to improve instruction, educators should make data analysis a part of an ongoing cycle of instructional 
improvement (Hamilton, Halverson, Jackson, Mandinach, Supovitz, & Wayman, 2009). Data use should not be 
relegated to a periodic event but should become a part of regular practice in schools and districts. Data use should 
also be prevalent across all levels of the school system, including classrooms, professional learning communities 
(PLCs), schools, districts, and state departments of education. Data use can be effective for many activities, including 
improving teaching strategies within classrooms, updating departmental priorities, and altering the educational climate 
within a school, district, or state (Wade, 2001). 

High-performing systems utilize data to improve instruction through a number of practices, including 

• reviewing data during weekly collaboration time, 

• holding data discussions during daily team instructional planning times, 

• dismissing students early periodically to allow teachers uninterrupted time for team data discussions, 

• holding all-staff meetings during which data is discussed, 

• providing teachers with a few hours of dedicated data review time each week through the use of substitute 
teachers or support personnel, and 

• discussing data collaboratively with other schools and districts. (Datnow, Park, & Wohlstetter, 2007) 

How Does a School or District Begin? 

An organization seeking to use data for informed decision making in order to improve student achievement should 
begin by considering who will lead the efforts within the school or district. Then, the organization needs to begin 
to create a culture that values data and the resulting information, while also familiarizing data teams with the data 
inquiry process. 

When beginning to implement the data inquiry process, it’s a good idea to start small, with questions that can be 
answered relatively quickly and easily. Beginning with a small question that can be analyzed and followed with simple 
action, leading to a “quick win,” will help to build support for data use within the community. 

The data analysis process is much like a sport; with practice over time, it will become easier until, eventually, the steps 
in the process and associated language will become second nature to users. When the process becomes frustrating or 
difficult, it is the responsibility of the data team members to push and support themselves and their teammates. As 
the team experiences successes and student achievement climbs, it will become easier to make the connection between 
the team’s hard work and the positive outcomes it creates. 
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Data Teams 


The full data analysis process, when conducted effectively, is a collaborative endeavor. Through collaboration, 
teams can amass the resources and manpower needed to collect and organize data and numerous points of view on 
questions, analysis, and action. 

Before a school or district begins to organize data around identified questions, it is important to identify a group of 
individuals within the organization who will hold primary responsibility for organizing, analyzing, and coordinating 
responses informed by the data. This group becomes the data team. 

What Is a Data Team? 

Data teams are interdisciplinary groups that field challenges, questions, and insights from the greater organizational 
community. They are collaborative and work together to monitor student and school performance and efforts to 
improve performance within the organization (White, 2005). 

The way that an organization manages its data teams can vary. Sometimes, schools may choose to support numerous 
data teams, delineated by subject, grade level, or specialty. Alternatively, schools might choose to have a single guiding 
team that works with challenges and related data at all levels of the school. Some schools might combine both 
approaches, with a large, school-wide team that works with smaller, more specified teams. Districts often pursue a 
systemic approach, with an overarching data team at the district level that works with school-wide teams and school- 
level subject and content data teams (Love, Haley-Speca, & Reed, 2010). Sometimes, individual teachers may form 
single-person data teams. The way that a school or district chooses to organize and manage its data teams will depend 
on the context, including the school’s goals and available resources (Hamilton et al., 2009). 

Role & Function 

The educational data team’s role is to engage in data-driven dialogue in order to identify and solve student 
achievement problems, issues, challenges, or gaps within the school or district setting. The data team serves in an 
advisory capacity, as it guides selection of the questions for inquiry, collects and analyzes the data, and suggests 
actions that can be taken to improve on a situation within the educational environment. It is not the team’s 
responsibility to police data use within the school or district or to ensure that policies are being followed. 

The data team is responsible for encouraging and enabling a culture of data use and continuous improvement 
throughout the school or district. As indicators of improved achievement occur, it is likely that the data-based inquiry 
concept and associated behaviors will spread throughout the organization. 

Successful data-based decision making and action are more likely to happen at the classroom and grade levels when 
the efforts are supported by a data-informed culture at the school and district levels (Hamilton et al., 2009). In a study 
of data-driven schools, it was found that successful schools did not appoint a single person to hold responsibility 
for the school’s data and data analysis. In successful schools, all school-based personnel, particularly teachers and 
principals, were explicitly expected to understand and use data to improve instruction and achievement (Wohlstetter, 
Datnow, & Park, 2008). 

Structures & Processes 

Data teams can be comprised of endless combinations of teachers, administrators, staff, and other stakeholders. The 
team members should be dedicated to the process and its intended outcomes. To enable effective and efficient work, 
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practitioners suggest keeping membership to a maximum of 15 people (Learning Point Associates, 2004). If the team 
is larger, smaller groupings of pairs or trios can simplify communication patterns and allow for more efficient task 
completion (Wellman & Lipton, 2004). 

When a data team is in the early stages of development, the group should begin by taking the time necessary to build 
a well-functioning team. This will include establishing group norms, clarifying the group’s purpose and goals, and 
developing a monitoring and control process to continually revisit the quality of group interactions (Love, Haley- 
Speca, & Reed, 2010). 

Effective teams typically commit themselves to the following conditions when working together: 

• A climate of trust and openness 

• Open and honest communication 

• A sense of belonging 

• Diversity valued as an asset 

• Creativity and risk-taking 

• Ongoing evaluation with ability to self-correct 

• Members who are interdependent 

• A consensus decision-making style 

• Participatory leadership 

(Learning Point Associates, 2004) 

Some examples and tools for building and engaging data teams are included in the Data Team Logistics section of this 
guide (beginning on page 31). 

In studies of data analysis processes in education, data teams at successful schools met weekly for structured data 
discussion meetings. Within these schools, teachers were provided with daily grade-level planning time, and school- 
wide data meetings occurred routinely throughout the year. These school-wide meetings were particularly effective 
when they were coupled with early student dismissals, to allow for a few hours of focused analysis and discussion. 
Meetings were well-structured and participants were held to high expectations for preparedness and engagement 
(Wohlstetter, Datnow, & Park, 2008). 

Like the continuous improvement cycle it is meant to drive, the data team process is continuous. The data team’s 
work is never complete. The team should endeavor to continually improve instruction and achievement in an iterative 
process of measurement and implementation (Datnow, Park, & Wohlstetter, 2007). 

Ongoing Support Needs 

Data teams may require routine and ongoing professional development or training services from experts within the 
school, district, and/ or state department. The goal of these services is to provide training on data access and to help 
the data teams remain informed about changes and updates to data sources, data storage systems, and data-related 
processes at all levels of the organization. Ongoing support can also help the team to maintain fidelity with its goals 
and the goals of the system in which it operates (school, district, or state department). 
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The data team may also require ongoing support from the administration at the school, district, or state level, to assist 
with follow-through on the team’s findings. For example, if a school-based data team identifies a need for updated 
teaching practices to improve student achievement, assistance and support from the school’s administrator will be 
needed to get full commitment to the new practice. 

The data team will also need ready access to high-quality, relevant data. Team members will need to know how to 
locate and acquire data sets. It should also be clear to all users that the quality of the decisions made and actions taken 
are heavily dependent on the quality of the data used. This means that data collection and entry must be both timely 
and meticulous. The data team will also need a dedicated location for data storage. Depending on the size of the data 
sets, this could include anything from a large data warehouse to a single spreadsheet. 

A data infrastructure alone will not drive data-based decision making and action. Staff members will need to have the 
tools needed to conduct a process for using data to make decisions and plan action. Effective processes are structured 
and systematic, helping teams to stay focused on using data to inform action in specified areas. This guide outlines a 
systematic and comprehensive process for using data effectively to make decisions and implement actions aimed at 
improving student achievement. 
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Stage 1 : Ask Good Questions 

Corresponds to Toolkit Tools 1A, IB, 1C, page 38 


Organizing data around essential questions about student performance helps educators use data while maintaining a 
clear focus on student learning. The data team’s first task is to clarify the questions that need to be answered before data 
collection begins. 

It is important to have a focused target for inquiry, and teams need to spend time developing focus areas and 
questions. To assist teams in working through this stage effectively and efficiendy, a bank of common questions is 
available in Appendix B (page 66). 

Focus Area for Inquiry 

The data inquiry cycle begins with the identification of an overarching focus area, often by an administrator prior to 
convening the data team. If this is not the case, the data team will need to identify a focus area. 

The overarching focus area serves as an “umbrella” throughout the data inquiry cycle. A potentially infinite number of 
questions related to the focus area can then be asked. For example, within “improve student achievement,” a common 
focus area, the following questions might be asked: 

• Why have 4*-grade math scores declined over the last three years? 

• Is there a relationship between 6 th graders’ behavior incidents and the success of their transition to 7 th grade? 

• How do this year’s ll*-grade ACT scores compare to the scores from the past two years? 

• Has our new focus on credit recovery programs had any impact on graduation rates? 

These questions vary considerably in focus, context, and possible actions; however, they are all related to “improve 
student achievement.” 

Developing Questions for Inquiry 

The goal of this stage is to ask good questions that data can answer. While philosophical or theoretical questions may 
be interesting and important, they cannot be answered definitively through data. Examples of both types of questions 
are in Table 1 . 


Table 1 : Questions Data Can and Cannot Answer 


Data Can Answer 

Data Cannot Answer 

What is the relationship between ACT scores and college- 
going rates? 

What does achievement really mean? 

How are our scores trending in mathematics and science 
over the last three years? 

Is it more important for students to learn mathematics or 
science? 

What factors might be contributing to the increase in 
suspensions this year? 

Do standardized tests really matter? 


Identifying potential questions for analysis and narrowing them down to a single question can be a time-consuming 
and difficult process. If the focus of an individual question becomes too broad, however, the volume of data becomes 
overwhelming and there is a danger of not addressing key student learning or program needs. It is important to limit 
the scope of the questions by considering whether or not they are clear, measurable, and align with the focus area. 
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Other considerations include whether the question is something that the team can investigate within the provided 
timeframe, whether the data needed to investigate the question is available, and whether the question represents a 
pressing issue for the organization as a whole. 

There are three common types of questions that data teams might consider during this stage of the process (Table 2). 
Descriptive questions address the current state of things, relational questions focus on the relationship between sets 
of variables, and causal questions explore the impact that one variable may have on another (Trochim, 2006). 


Table 2: Common Types of Questions for Data Inquiry 


Type of 
Question 

Description 

Examples 

Descriptive 

Describes what is going on or 
what exists 

• How are 5 th graders performing in reading .this year? 

• How do School X’s SAT scores compare to School Y’s SAT scores 
in 2011? 

Relational 

Explores the relationship 
between two or more variables 

• What is the relationship between end-of-course grades and 
scores on the state test? 

• What is the correlation between ethnicity and enrollment in AP 
courses? 

Causal 

Determines whether one or 

more variables causes or ef- 
fects one or more outcome 

variables 

• What is the impact of direct instruction on kindergarteners’ pho- 
nemic awareness? 

• What is the impact of attendance on 4 ,h -grade MAP scores? 


Regardless of the type of question selected, the team should ensure that it is clear, straightforward, and sufficiently narrow 
to allow for an effective data inquiry process. Questions that are clear and straightforward use common language and are 
easily understood by persons outside of the team. Questions that are sufficiently narrow will allow the team to answer the 
question within a reasonable time frame and without requiring an excessive number of data sources and variables. 

A question that is too broad can require too many pieces of data and too much preparation and analysis time. This 
can result in a frustrated data team that is unable to complete the cycle of data inquiry in the allotted time. A question 
that is too narrow will not guide the team towards change that can positively affect the whole organization. Questions 
that are too narrow are typically those that consider only one or a very small number of students. Examples of overly 
broad and sufficiently narrow questions are listed in Table 3. 


Table 3: Overly Broad and Sufficiently Narrow Questions 


Overly Broad 

Sufficiently Narrow 

Is there any relationship between AP scores 

and other scores? 

In 2010 at Main High School, how did final scores on the 1 1 ,h -grade AP History 

test correlate to end-of-course grades? 

Is behavior related to outcomes? 

Are suspension rates related to high school graduation rates in our district? If so, 

in what way? 

What impacts MAP scores? 

How does student attendance affect 4 ,h -grade MAP scores? 


While the overly broad question may be interesting to ask and may be the ultimate goal over many periods of inquiry, 
the sufficiently narrow question allows the team to complete the data inquiry process in a reasonable amount of time. 
The sufficiently narrow question provides the team with clear direction for the scope of the question and the data that 
needs to be collected. 
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One way to determine whether the question is too broad is to itemize all of the variables inherent in the question. 
The variables are those data sources that will need to be examined in order to answer the question. When a question 
is too broad, the list of variables is quite long and likely overwhelming to the group. Examples of questions and the 
associated data are provided in Table 4. 


Table 4: Using Variables to Determine if a Question is Too Broad 


Question 

Variables/Types of Data Needed 

Overly Broad 

Is behavior related to outcomes? 

Behavior Data: suspensions, expulsions, detentions, calls home, all types of 
behavioral infractions 

Outcomes Data: graduation, dropout, end-of-course grades, GPA, college 
admittance, grade retention, remediation rates 

Time Period: pull data for an indeterminate number of years 

Sufficiently Narrow 

Are suspension rates related to high 
school graduation rates in our district 
over the last two years? 

Behavior Data: suspension rates at each high school in the district 

Outcomes Data: graduation rates at each high school in the district 

Time Period: pull data for just the last two years 


In the first example, there are coundess variables and data sources needed to answer the question. The team may 
become frustrated by all of the possible combinations of data as well as the time and effort necessary to collect and 
analyze each of them. 

In the second example, the question has been narrowed to include only a handful of variables and data sources. With 
this smaller dataset, the team can organize, review, and analyze the data relatively quickly, answering its focus question 
in a reasonable amount of time and moving forward to take action. 

If the team chooses at a later time to continue on this path of inquiry, they may expand the question to consider 
additional years, grade levels, courses, or schools. 

The Question Cycle 

Data teams will often develop numerous possible questions for inquiry during the initial focused brainstorming 
session, and additional questions are likely to emerge during the course of data analysis. 

It is important to document these questions for potential use in upcoming data cycles. Keeping such a log honors the 
input of all team members and will accelerate the process of asking good questions during the next cycle of inquiry. 

A Parking Lot Tool (7 A, page 47) is included in the Data Inquiry Toolkit to assist with the recording of questions and 
other ideas (page 64). 

After identifying a good question for inquiry, the team will be ready to identify, collect, and prepare the data needed to 
answer the question. The question should serve as a filter that helps to identify which sources of data are relevant for 
the inquiry. 
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Stage 2: Collect and Prepare the Data 

Corresponds to Toolkit Tool 2A, page 39 


Data collection is simply the compiling of data that address the focus question. Data 
collection is an important part of the data inquiry and decision making process. It should 
be a planned and purposeful process, as valuable data will guide the development of 
improvement goals that benefit all students. 

Teams may discover that the data needed to answer the focus question is not readily available. 
This is powerful information to provide to the school-, district-, and state-level technology 
and data teams as they continue to build and refine longitudinal student data systems. When 
the required data is not readily available, the team can either select another question from the 
initial set or begin brainstorming new questions for which data is more available. 

Data Levels 

Data is available at various levels of the organization. From the broadest, most aggregated, 
sources to the most specific sources, data can typically be collected as identified in Figure 1 . 
There may be additional levels available in regional or national systems, as well. 

A data team may convene at any level of an educational organization to conduct a cycle of 
data inquiry. For example, a state -level team might analyze data on all schools, districts, or 
intermediate agencies within the state; an intermediate agency might consider data only on 
those schools with which it partners; and district and school teams might consider more 
specific school or classroom data. 

Types of Data 

Common types of data available in an educational environment include demographic, 
perceptual, performance, and program data, as illustrated in Figure 2. 

Demographic Data 

• This type of data includes descriptors of students, including gender, ethnicity, 
socioeconomic status, and descriptors of the organization, including enrollment and 
attendance. 

• Typically, it is used as a way to disaggregate, or pull apart, the other focus data. 
Demographic information can allow the team to look at a question for different 
subgroups of students. This data may also be useful if the team has convened to solve 
an issue of equity, such as differing math performance between boys and girls. If, 

for example, a team is exploring a relationship between an instructional practice and 
reading outcomes, the team could consider whether the instructional practice leads 
to increased reading skills for all students or for different genders, ages, ethnicities, 
socioeconomic status levels, or other groupings of students. This data can give nuance 
to the question and perhaps focus later action. 



Intermediate 

Agency 


District 


School 


Classroom 


Subgroups 


Figure 1 : Common 
Levels of Education Data 
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Demographic 


Perceptual 


Performance 


Program 



Figure 2: Common Types of Educational Data 


• It might also be collected through the use of surveys or questionnaires (which are often associated with perception 
data), to better understand the descriptors of students’ out-of-school lives. For example, teams may want to 
consider relationships between achievement and students’ bedtimes, how many people live in the home, or how far 
students have to travel to get to school. 

Perceptual Data 

• This type of data includes stakeholder surveys or questionnaires (student, parent, staff, community member, 
graduate) and observations. 

• Provides information on how stakeholders feel or what they observe about the organization or its activities. A 
team might use this data to identify correlations between stakeholder perceptions and the question for inquiry. For 
example, a team might consider whether there is a correlation between a student’s ACT scores and his parent’s 

or guardian’s perception of school quality. Key stakeholders in a school or district include students, parents, staff, 
alumni, and community members. 

• This data is often collected through surveys or questionnaires and may require collection after the team has 
identified a goal and a question for inquiry, to ensure a clear connection between the data and the inquiry. 

Performance Data includes both Achievement Data and Outcomes Data. 

Achievement Data 

• This type of data includes state assessments, district and school assessments, curriculum-based assessments, 
interim and benchmark assessments, end-of-course grades, and grade point averages (GPA). 

• It is useful whenever the team wants to improve and measure student achievement. For example, it would be used 
if the team wants to understand more fully the relationship between MAP scores and NeSA scores. 

• When considering education data, unless the focus question calls for a specific type of achievement data (for 
example, MAP scores), the team should be careful to avoid overreliance on any single source of achievement data. 
When considering test scores within the context of answering the focus question, the team should try to use at 
least two types of achievement data, to ensure that the question is answered with appropriate accuracy and to avoid 
skewing the response. For example, if the data team wants to know if a certain instructional strategy is correlated 
with increases in student reading skills, they may want to consider reading scores on multiple assessments, 
including classroom-based and state standardized tests. 
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Outcomes Data 

• This type of data includes anything that measures final results or conclusions for students. Examples are 
graduation rates, dropout rates, mobility rates, suspensions and expulsions, behavior rates, remediation rates, 
college acceptance and attendance, and career readiness. This data can also include long-term outcomes, such as 
career achievement, personal milestones (such as marriage or childbearing), or mortality rates. 

• It is most useful to a team that wants to consider the effects of a variable on what happens to students within a 
specific time period or at an end-point in their education. For example, a team may want to consider whether there 
is a correlation between 6 th -grade math achievement and college acceptance rates, to determine how the K— 12 
school environment can improve student outcomes. 

Program Data 

• This type of data includes descriptive data about how education and related activities are conducted within the 
organization, including, for example, the textbooks used, the levels of staffing or professional development at 
the school, the schedule of classes, curricular sequences, instructional strategies, the nature and frequency of 
assessments, extracurricular activities, and even the school setting. 

• It can be useful to teams considering the effects of systemic factors on student outcomes or achievement. For 
example, a team might investigate whether a correlation exists between a teacher’s years of service and students’ 
achievement on the NeSA. 

Having the right data is very important. The data team will frame its work when it chooses which data to collect and 
analyze. The data must direcdy link to the question under inquiry; the data team cannot answer a question and develop 
actionable steps with unrelated data. 

This is not an exhaustive or comprehensive list of the types, sources, or potential uses of data and should be used 
only as a guide. Educational organizations have an abundance of data available, and it is the data team or data user 
who will determine which sources are most useful to answer the question at hand. Data become informative and 
meaningful as they are used and analyzed. 

Collecting the Data 

Once the team identifies the data needed to answer the focus question, the next step is to determine where the data 
are located and how to acquire the data. In some cases, the data will be stored at the classroom level (for example, in- 
class formative assessments or interim course grades). 

In most cases, data are available at the school, district, or state levels. Some organizations store student data 
consistently in a single school-, district-, or state-wide longitudinal data system or student management system. Other 
organizations store data in building-specific systems. The organization’s data storage systems and their associated 
permissions structures will vary widely and will determine the complexity of the data gathering process. 

One or more team members should be given the task of collecting the necessary data prior to the next meeting. Data 
access, like data storage, will vary depending on the nature of the data required. If it is not publicly available, the 
data may need to be acquired through multiple avenues, including from classroom teachers, building, district, or state 
administrators, or data managers. 

Grouping relevant data sources together ensures that the team is comparing data sources appropriately. When 
analyzing the data for correlations, causal relationships, or other interactions, it is important to keep data sources 
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grouped according to relevant year, location or other important variables. For example, if the team aims to consider 
the correlation between NeSA scores and attendance, it cannot use data from two different years. The team will need 
to compare 2010 NeSA scores to 2010 attendance rates in order to accurately answer the question. 

If the team discovers at this stage that the data are unavailable or inaccessible, they will need to consider whether they 
can answer the focus question based on what is available. If not, the team should reconvene to identify a new question 
for inquiry. The team should also provide feedback to data managers to ensure that the need for this data is built into 
the system for future cycles of inquiry. 

Data Preparation 

Next, the team should organize the data into charts, graphs, and tables, in order for productive analysis to occur. 
Preparing the data is an important step, as it helps the team to visualize the facts and patterns in the data in an 
efficient manner. In order to maintain effective meeting outcomes, the data should be prepared ahead of the next 
team meeting. The team should not wait until Stage 3 to complete data preparation activities. 

Depending on the types of data collected, teams might choose to use graphs, tables, color-coding, or other methods 
of displaying the data. Unless student names and identifying information need to be included for an accurate analysis, 
all student-level data should be presented using identification numbers in place of names. 

Graphs provide a way to easily identify trends or differences between groups. Some graphs will also help the team to 
visualize changes over time or correlations between two variables. Common types of graphs are described in Table 5. 
Further information on common graphs is in Appendix D (page 69). 


Table 5: Different Types of Graphs and When to Use Them 


Graph Type 

Best Uses 

Line Graph 

• Tracking changes or trends over a period of time 

• Comparing two groups over the same period of time 

Example: Scores earned by 5 th graders on the last 10 math tests 

Bar Graph 

• Comparing a variable across different groups 

• Comparing two groups on a single variable 

Example: Average attendance rates by grade level 

Pie Chart 

• Comparing parts of a whole 

• Comparing percentages 

Example: Breakdown of the school’s teachers’ years of service 

Scatterplot 

• Identifying relationships or correlations between two variables 

• Identifying relationships in large data sets 

Example: Correlation between classroom grades and ACT scores 


Color-coding is a method of quickly identifying trends and patterns in a spreadsheet through the use of color. 
Teams often will code “bad” data in red and “good” data in green. For example, when considering the proficiency 
levels of a group of students, those who scored as non-proficient might be colored red, so that the team can 
quickly make an estimate of the percentage of students in this category. An example of color-coding is available in 
Appendix C (page 68). 
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Stage 3: Find Trends and Make Observations 

Corresponds to Toolkit Tool 3A, page 40 


Once the data has been collected and organized, the team should be ready to consider what the data says. It is 
important to first focus on the factual evidence before considering context. This will help the team to avoid making 
preemptive conclusions, excuses, and assumptions about the challenge or issue in question. 

Making excuses is a common approach to explain why students are not achieving at higher levels, which is often 
something that is completely outside of the school’s control (e.g. environmental factors). A purpose of the data 
analysis process is to move beyond those assumptions and to identify areas the team can control and actions the team 
can take to make positive changes in student achievement. 

During this stage, it is helpful for team members to use computers to sort and reorganize the data to enable 
efficient disaggregation. All visual displays of data, including spreadsheets, graphs, and color-coded tables, should 
be completed before beginning this stage. Though the data may need to be resorted, recalculated, or presented in a 
different format, it is not recommended that the group wait until the Stage 3 meeting to complete the bulk of the data 
preparation. 

Find Meaningful Trends and Patterns In the Data 

Data team members should keep the focus question in mind as they review the data and search for patterns. To 
identify the trends and patterns, the team can make calculations, disaggregate the data, and review visual displays of 
the data. 

Make Calculations. A quick method for finding meaning in the data is counting, adding, dividing, or averaging 
the data. 

Counting: In counting, a team can determine how many members of a group were included in a particular 
variable. For example, counting will inform a team that 37 6 th graders had grades of B or higher in mathematics. 

Adding: In adding, a team can determine how many members of multiple groups were included in a particular 
variable. For example, adding will inform a team that a total of 64 middle school students had grades of B or 
higher in mathematics. 

Dividing: Dividing will provide percentages, allowing a team to compare across groups. This method can be 
particularly helpful when groups are of different sizes. For example, by dividing the number of students with 
a grade of B or higher by the total number of students in the group, the team can determine that 46% of 6* 
graders and 58% of 7 th graders had grades of B or higher in mathematics. 

Averaging: Determining the average for a group will help the team identify whether the mean is where it needs 
to be and will allow the team to compare means across groups. For example, averaging will tell a team that the 
mean math grades were B- in 6 th grade, A- in 7 th grade, and B in 8 th grade. 

Disaggregation. By breaking the data into subgroups, the team can consider more distinct patterns related to the 
focus question. The team can use the same calculations described above, but apply them to more specified groupings. 
The team can also sort a full data set according to the disaggregation variables. For example, the team might 
determine that, in mathematics, the average male 6 th grader receives a B but the average female 6 th grader receives 
a C+. This gives them more nuanced information about math performance. This pattern may help the team to 
formulate actions in the later stages of the process. 
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ID 

Grade 

Gender 

Math Grade 

66667 

6 

F 

73 

409272 

6 

F 

76 

251753 

6 

F 

87 

485833 

6 

F 

76 

868004 

6 

F 

81 

530154 

6 

M 

90 

115447 

6 

M 

89 

222288 

6 

M 

73 

750996 

6 

M 

79 

660644 

6 

M 

90 

927193 

7 

F 

87 

896362 

7 

F 

74 

191763 

7 

M 

92 

945834 

7 

M 

87 

150628 

8 

F 

77 

395946 

8 

F 

80 

571100 

8 

F 

87 

496242 

8 

M 

88 

721724 

8 

M 

72 


By sorting the data table by 
grade, then by gender, the team 
is able to quickly calculate the 
average scores for 6 th -grade boys 
and girls. 

Once the team knows that boys 
average 84 while girls average 
79, they may be able to provide 
a more targeted action approach 
to improving the school’s 
mathematics scores. 


Figure 3: Example of Sorting for Disaggregation 


Some common school-level indicators upon which teams can disaggregate data include: 

• Grade level 

• Ethnicity 

• Gender 

• Special Education status 

• Socioeconomic status 

• English Language Learner status 

• Teacher’s years of service 

• Student’s percentile rank 
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Visual Displays. Visual displays can be very powerful in helping the team identify patterns in the data. These should 
be prepared ahead of time and will often take the form of graphs, tables, or color-coded figures. These displays can 
help patterns appear more readily to team members. Examples of visual displays are available in Appendices C and D 
(pages 68—72). 

If the team identifies trends or patterns that are unrelated to the focus question but may merit review at a later 
time, they should note this in the team’s Parking Lot (page 66) and plan to return to them at the close of the 
current project. 

Make Factual Observations About the Data 

Once the team has begun to notice trends and patterns, it is important to record clear, factual observations about 
the strengths and the challenges evident in the data. The team should avoid explaining the patterns in the data at 
this stage. 

When analyzing the data, the team should first consider the stories that are told by each type of data alone, then by 
the disaggregated and combined data sources. For example, if the team has school-wide data for NeSA-M scores and 
attendance rates for 2011, they should consider the NeSA-M data, then the attendance data, followed by the two data 
sets combined. Then the data sets can be considered in disaggregated ways, such as looking at the data by grade level, 
by gender, or by free-reduced lunch (FRL) status. Quickly, even a small number of data sets can become very rich in 
information. 

Team members should avoid introducing bias and interpretation in their observations. Bias is evident in statements 
that indicate assumptions or prejudice about something, someone, or a group of people. Interpretation is evident 
in statements that attempt to explain something or the meaning of something. While interpretive statements might 
actually turn out to be true, they are not fully based on what the data shows. Examples of each are provided in Table 6. 


Table 6: Examples of Bias and Interpretation 


Type 

Example Statements 

Bias 

• Boys are better than girls at mathematics. 

• Students living in poverty will achieve at lower levels. 

• Foreign-born teachers are not as good as American teachers at teaching 
English Language Arts. 

Interpretation 

• Fourth graders are having behavior issues because they play too many 
video games. 

• The ACT scores have been dropping because we moved mathematics to 
the morning. 

• Gym class helps students do better in afternoon classes. 
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One way the team can ensure factual observations about data is to make SURF statements (Figure 4). Examples of 
statements that adhere to SURF appear in Table 7. 


Specific 


Understandable 


Related 


Factual 


Statements comment on only one item in the data. These statements 
are straightforward and clear because they involve single sources. 

Statements are easy to comprehend even without having intimate 
knowledge of the data. 

Statements are those that are relevant to the question the team is 
seeking to answer. 

Statements only incorporate the data; they do not involve 
assumptions, inferences, or biases. 


Figure 4: Descriptions of SURF Statements 


Table 7: SURF Data Statements 


Example Focus Question: How are 4 ,h -grade attendance rates related to 4 th -grade MAP scores? 


Not SURF 

SURF 

Specific 

Fourth graders have an overall 
attendance rate of 76%; they aren’t 
doing well on the MAP; the boys have 
better attendance than the girls. 

The overall attendance rate for 4 th - 
grade students at ABC School in 

201 1 was 76%. 

Not SURF because the statement considers numerous ideas and data sources. It is not specific to a single fact. 

Understandable 

Based on the MAP data set’s trend 
lines, we can see that the kids 
weren't doing that well. 

In 201 1 , 24% of 4 th graders at ABC 
School scored in the 50 th percentile or 
higher on the MAP test. 

Not SURF because the statement references data and graphs that are not known to individuals outside of the group and 

does not make a specific statement of fact. 

Related 

Fourth graders at ABC School had 
the highest number of behavior 
incidences of any group of students 
in 2011. 

In the 201 1 4 th grade at ABC School, 
boys had an average attendance rate 
of 73% while girls had an average 
attendance rate of 81 %. 

Not SURF because the statement does not provide a fact with relevance to the focus question. 

Factual 

Fourth graders’ attendance rate of 

76% in 201 1 was due to the fact that 
the local factory laid off 100 workers, 
so most kids couldn’t get reliable 
transportation to school. 

In 201 1 , the 3% of students who 
scored in the 75 th percentile or above 
on the MAP test also had attendance 

rates of 98% or better. 

Not SURF because the statement introduces interpretation and attempts to provide reasons for the fact presented in the data. 
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The team should record all of the SURF statements and hold each other accountable to the requirements of SURF. 
Statements that are data-based but include interpretation or context should be held over until the next stage. 

This can be one of the most exciting stages of the data process, but it can also be one of the most time-intensive. If 
the team has time constraints, it might be best to set time limits for each data source. 

After the team has thoroughly examined the various data sources, uncovering meaningful patterns, relationships, and 
facts that are related to the focus question, they are ready to move on to Stage 4, during which they will interpret the data. 


Stage 4: Interpret the Data 

Corresponds to Toolkit Tools 4A, 4B, 4C, 4D, and 4E, pages 41-43 


During Stage 3, the data team considered only factual observations that were readily observed in the data patterns and 
trends. During Stage 4, the team begins to interpret those patterns in both the context of the organization and the focus 
question. Then, the team can answer the focus question and develop hypotheses to drive subsequent action. 

Identify Strengths and Challenges 

The team will begin by reviewing the factual statements about the data. At this stage, the group can introduce 
interpretation of the facts and statements, placing the statements into the appropriate context. Though data- 
based interpretive statements are allowed at this stage, statements grounded in bias are rarely helpful and should 
not be included. 

During this activity, the team will begin to identify areas of strength and challenge that have emerged from the 
data. It’s important for the team to take time to recognize and celebrate strengths. Not only does this provide a 
positive counterpoint to the challenges, but the exercise of identifying strengths can help to ensure that they are 
sustained over time and, when possible, applied to future improvement efforts. 

It may be that one or more of the observed strengths can be applied to the current focus area. For example, 
while investigating a focus question dealing with middle school math achievement, the team may discover that, 
over the last three years, students have shown growth in achievement between the 6 th and 7 th grades. If the team 
can identify why that growth is occurring, it may inform action for other grade levels. 

Once strengths have been identified and celebrated, team members will begin identifying challenges. Challenges 
represent factors that may be impeding improvement or positive outcomes in student achievement and 
performance. They are not barriers that were experienced during the cycle of data inquiry, such as a lack of data 
access or difficulty scheduling meeting times. 

It is likely that multiple challenges will be identified through the data. Some challenges, however, will have a 
higher priority for action than others. To identify these, the team will identify all of the observed challenges, 
ensuring that they are related to the overarching focus area and focus question. Unrelated challenges should not 
be forgotten but should be reserved for a later cycle of data inquiry. 
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Sometimes, the team may find that the data indicates that there are no challenges in the particular area being 
reviewed. This is valuable information and something to celebrate. If this happens, the team should share its 
findings with the greater community and move on to a new focus question or additional data sets. 

Prioritize the Challenges and Choose a Key Challenge 

Once the challenges are listed, the team should prioritize and rank them to identify the challenge that will be the focus 
of action in later stages of the cycle of inquiry. The ranking process can occur in a number of ways. Team members 
can be asked to identify their top three challenges, their top challenge, or to rank all challenges. It may be important to 
remind the team that lower-rank challenges can be pursued later. 

When ranking the challenges, team members should consider whether the challenge exists within the team’s area of 
control — and, therefore, is something that the group can directly change. Other challenges may exist within the area of 
concern, where the group has no ability to alter any factors related to the challenge, or the area of influence, where the 
group can change factors around the challenge but not the challenge itself (Covey, 1989). Instead of focusing on conditions 
over which they have little or no control, proactive educators will focus their time and energy on things they can control. 


Table 8: Examples of Areas of Concern, Influence, and Control 


Example Focus Question: What is causing ABC School’s low 4 ,h -grade MAP scores? 

Area 

Example 

Reason 

Concern 

Parents aren’t enforcing early 
bedtimes, so students are tired when 
they take their MAP tests. 

There is nothing the team can do to 
force students to go to bed earlier. 

Influence 

Parents aren’t encouraging students 
to do well on the tests. 

The team can send flyers home and 
hold information nights to inform 
parents of the importance of the MAP 
tests, in an attempt to influence the 
parents to encourage their children. 

But the team cannot force parents to 
encourage their students to do well. 

Control 

Most of the 4 ,h -grade students are 
arriving at school not having eaten 
breakfast, so they are hungry when 
they take their MAP tests. 

The team can take action to provide 
a healthy breakfast for students on 
testing days. 


The ranking process may result in selection of challenges that are only within the area of influence — and none within 
the area of control. If the team believes that it has sufficient influence to warrant action in that area, it is acceptable to 
choose challenges within the area of influence. 

Regardless of the method used to rank challenges, the team’s leader should ensure that there is consensus among the 
group and that all members feel comfortable pursuing the identified top- priority challenge. While it may not be possible 
for the team to agree completely, it is important that all members understand and are committed to the chosen challenge. 

At this point, it is important for the team’s facilitator to ensure that everyone has a shared understanding of and agree 
with the process and decisions to date. 
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Identify Driving Factors for the Challenge 

Once a key challenge has been identified for further inquiry and action, team members transition to 
hypothesizing reasons for the challenge’s existence. These reasons, known as driving factors, will become the 
targets for action in later cycle stages. 

One way to identify driving factors for the key challenge is to ask a series of “Why. . .?” questions followed by 
“Because. . .” responses, as illustrated in Table 9. In this activity, the team considers reasons for the challenge by 
making interpretations and educated guesses and determining whether the driving factor is within the team’s area 
of concern, influence, or control. Finally, the team tests its hypotheses to ensure that the inherent assumptions are 
correct and supported by data. 


Table 9: Why/Because Driving Factor Identification Process 


Key Challenge: 

10 th -grade students have the highest suspension rates of all of the students in the school 

Why? 

Because... (Driving Factors) 

Which Area? 

Why do they have high suspension 
rates? 

They have a large number of 
behavior incidences. 

(^ConcerrT) Influence Control 

Why do they have so many behavior 
incidences? 

They have more free time in which 
to misbehave. 

y 

Concern (Influence^) Control 

Why do they have more time to misbe- 
have? 

They have lunch periods free. 

Concern Influence ((Control^) 


Another method of identifying driving factors is the Cause and Effect or Fishbone Diagram (Figure 5). This 
diagramming method encourages the team to consider numerous potential driving factors and to think broadly 
(George, Rowlands, Price, & Maxey, 2005). The team will place the key challenge in the diagram’s “head” and provide 
numerous boxes for categories of driving factors. The team begins by brainstorming what the categories should be, 
and then begins to drill down into specific causes, or driving factors, of the key challenge (George et al., 2005). 
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Environmental Factors School Culture 



10 ,h -grade students 
have the highest 
suspension rates in 
the school. 


Figure 5: Fishbone Diagram of Driving Factors 


The fishbone diagram can lead to more specific ideas and might be best used for more complex issues, to help the 
team organize the various driving factors that might be present. 

Once potential driving factors have been identified, it’s important to check that the assumptions inherent in the 
hypotheses are correct. In the example from Table 9, the team will want to consider whether the assumptions “10 th 
graders have a large number of behavior incidents” and “10 th graders have more free time” are correct. A quick look 
at the relevant data will reveal whether these assumptions are supported by data. If they are, the hypotheses can 
remain. If they are not, the hypotheses should be rejected and other potential driving factors should be considered. 

Choose a Top Driving Factor for the Key Challenge 

Ultimately, the team will attempt to have measurable impact on the key challenge and the focus area by taking action 
to improve or eradicate the driving factors. In order to focus efforts, the team will choose a single driving factor upon 
which to act. This factor should be within the team’s area of control or, if that is not possible, the area of influence. 

The team should rank order the controllable driving factors according to the impact each factor has on the key 
challenge. Factor 1 will have the largest impact. 

Once the team has chosen a top driving factor for action, it is ready to move on to Stage 5. 
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Stage 5: Plan for Action 

Corresponds to Toolkit Tools 5A, 5B, page 44 


A data-driven decision-making continuum begins with raw data and ends with meaningful knowledge that is used 
to make decisions (Breiter & Light, 2006). Educators should be aware that data-driven decision making does not 
guarantee effective decision making (Marsh, Pane, & Hamilton, 2006). Simply collecting, organizing, analyzing, 
and interpreting data does not ensure that the data will be used to drive decisions and action or lead to school 
improvement. 

Attention must be given to planning actions based on data, which is typically more challenging than the act of 
reviewing data (Marsh, Pane, & Hamilton, 2006). The data inquiry process ensures that the data team and school 
or district community at large bases its actions and strategies on facts, as opposed to uninformed guesses, when 
determining causal factors. 

After the members of the data team have collected, organized, and analyzed the data to identify patterns, challenges, 
and driving factors, they are ready to move to action planning. The goal of the full cycle of inquiry is to take action on 
the driving factors to positively affect the key challenge and ultimately improve the overarching focus area. 

To review, the four stages of the Cycle of Data Inquiry identify the focus area, focus question, key challenge, and top 
driving factor. Actions that improve or eliminate the top driving factor can be expected to improve or eliminate the 
key challenge, allowing the group to move toward improvement in the focus area. 

During this process, it is important for data team members to commit to using data-based facts for the purpose of 
driving inquiry and action. Assumptions are often made about how or why things are, which can lead to excuse-making 
or becoming convinced that the problem cannot be solved. It’s also tempting to analyze data or rationalize solutions 
before good questions have been asked. Following a prescribed process will ensure that, by the time the team reaches 
Stage 5, the decisions and resulting actions will be based on facts. 

Once the team has reached Stage 5, it is time to review the progress that has been made, to ensure that all members 
understand the decisions and observations made thus far. Keeping a running log of the cycle on easel paper or on 
a computer file allows the team to easily access and review it (a blank log is available as Organizational Tool 7B, 
page 47). 

Set a Strategy 

Once the data team has assembled actionable information from the data, it will need to design a specific strategy to 
improve the top driving factor, key challenge, and overarching focus area (Boudett, City, & Murname, 2007; Datnow 
et al. 2007; Halverson, Grigg, Prichett, & Thomas, 2007; Learning Point Associates, 2004; Marsh, Pane, & Hamilton, 
2006; Mid-continent Research for Education and Learning, 2010). 

The strategy will provide the team with a clear outline of the following: 

• The baseline measurement of the key challenge 

• The improvement the team would like to see in the key challenge 

• A deadline for improvement to have occurred 

• The specific action the team will implement to create this improvement 
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The first step is to consider what the data indicated about the key challenge and what change the team wants to see. 
The baseline of the strategic statement is formed by the baseline measurement, or the information that the data 
provided about the key challenge. The strategic goal statement for action will elucidate the improvement the team 
would like to see in the key challenge. Examples of this are in Table 10. 


Table 10: Developing the Strategic Goal Statement 


Key Challenge 

Baseline 

Strategic Goal Statement 

7 th graders have the highest rate of 
suspensions. 

The 7 th grade had 47 suspensions last 
school year. 

Reduce 7 ,h -grade suspensions by 

20%. 

Elementary math scores have been 
trending downward over the last three 
years. 

In 2010-201 1 , 47% of 4 th graders 
were proficient in mathematics. 

Increase 4 ,h -grade proficiency to at 
least 60%. 

The graduation rate is trending below 
that of neighboring districts. 

The 2010 and 2011 graduation rates 
were 67% and 65%. 

Improve the high school graduation 
rate to at least 76%. 


Next, the team should set an end point or deadline for measuring whether the desired improvement has been reached 
or is progressing. If the strategy does not appear to be having a measurable effect on the top driving factor or key 
challenge by this date, the team will have the tools it needs to reassess whether the strategy should be continued, 
amended, or set aside for another plan of action. 

After the team has set its end point, it should consider actions that might improve the driving factor, key challenge, 
and focus area. For example, if the strategic goal is to reduce 7 th -grade suspensions by 20 percent and the top 
driving factor has been identified as the passing periods that are introduced in the 7* grade, the team might decide to 
eliminate passing periods or assign teachers to monitor the hallways during these time periods. 

Once the team has completed its brainstorming, team members should review each potential strategic action and 
consider the following: 

• Relevance: Will it impact the top driving factor? 

• Time: Is it possible given the team’s time limit? 

• Resources: Is it possible given the team’s resources? 

The team should then prioritize the potential strategies. Those strategies that received “no” answers to any of the 
three considerations of relevance, time, and resources should not be ranked. The top priority strategy should be the 
one that the team agrees could most affect the key challenge and can be implemented within the given timeframe and 
resources. An example of this process is in Table 1 1 . 
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Table 1 1 : Example of Strategy Setting 


Focus Area: Improve ABC School's academic achievement 

as measured by state tests and end-of-course grades. 

Focus Question: What is the relationship between suspensions and achievement at ABC School? 

Key Challenge: At ABC School, 7 ,h -grade students have the highest rate of suspensions, with 47 total suspensions. 

Top Driving Factor: Students begin changing classrooms between periods at this grade level. 

Strategic Target: Reduce the number of suspensions by 20%. 

Potential Strategies 

Priority 

Ranking 

Is It Possible Given: 

Time? 

Resources? 

Change the schedule so that 7 th -grade students remain in 
the same classrooms and teachers rotate. 

3 

Yes 

Maybe 

Identify 8 ,h -grade peer leaders to keep watch in the 
hallways and to encourage the 7 th graders to behave 
better. 

1 

Yes 

Yes 

Hire hall monitors to keep watch and stop behavior 
incidents immediately, before they can escalate to 
suspensions. 

~~ 

Maybe 

No 

Require students to walk in single-file lines, silently, during 
class changes. 

2 

Yes 

Yes 

Strategy for Action: Identify 8 ,h -grade peer leaders to keep watch in the hallways and to encourage the 7 th graders to 
behave. 


In the example from Table 1 1 , the team brainstormed four potential strategies, all of which could impact the number 
of suspensions the school would experience the following year. Based on the team’s assessment of relevance, time, 
and resources, however, it decided to implement the second strategy. 

The chosen top priority strategy may not be the only strategy the team implements to improve the key challenge. The 
team should pursue one strategy at a time, however, to ensure strong implementation before moving on to a new 
strategy. Other potential strategies should be tracked in the Parking Lot (Tool 7 A, page 47) for implementation at a 
later date. 

Strategies must be clearly expressed to ensure consistent and faithful implementation. This will help the team to get its 
members and the community at large to commit to the plan of action and the changes that may be needed to realize 
the ultimate goal. 
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Clearly written strategic statements often take this form: 


By 

(date) 


, improve 


(key challenge) 


from 

(current data point) 


to 


by 

(future data point) 


(strategy) 


In the example from Table 1 1 , the strategic statement might be: 


By the end of the next school year , improve the 7 lh -grade suspension rate from a total of 47 in a school year to 
a total of 37 or fewer in a school year by identifying 8 th -grade peer leaders to keep watch in the hallways and to 
encourage the 7 th graders to behave better. 


This statement conveys important information: the timeframe, the deadline, the strategic goal, and the process by 
which the team expects to accomplish the goal. Everyone on the team should agree with the strategic statement, 
because it will drive the action steps in Stage 6. 
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Stage 6: Implement, Monitor, and Sustain 

Corresponds to Toolkit Tools 6A, 6B, pages 45-46 


The final stage of the cycle of data inquiry involves developing and implementing a plan for action, evaluating the 
effectiveness of that action, and either working to sustain the positive outcomes of the action or planning for new 
action. 

The duration of this stage can vary from a few weeks to months or even years. The amount of effort involved 
in initiating, monitoring, and sustaining the actions should be proportional to the scope of the strategy. In an 
organization that pursues frequent and regular cycles of data inquiry that produce new strategies and plans of action 
every few weeks, large amounts of time cannot be spent in monitoring the implementation of each strategy. But in an 
organization that pursues a single strategy for large-scale change over an extended period of time, it will be essential 
to invest more time and effort into ensuring that the single strategy is working and is implemented with fidelity over 
the long term. 

Prepare to Act 

As the team considers the actions it will take to implement the strategy, it is important to remember that the action 
plan should be manageable for all persons involved in the process. Ideally, the data team will involve all affected 
stakeholders (e.g, teachers, counselors, administrators) in the action planning, to ensure that they all agree with the 
plan and are committed to it. 

In preparing to act, the team will identify all of the activities needed to implement the designated strategy, which 
will result in an action plan, or a project plan (Tool 6A, page 45). The action plan ensures that all necessary steps are 
included and consists of the following pieces of information: 

Action Steps: The activities that need to be completed to implement the strategy 

Responsibilities: Identification of the team members and/or other stakeholders responsible for each of the 
action steps 

Deadline: The date by which each action step should be completed 

Resources: Identification of any specific materials or resources that will be needed to complete the action steps 

Some action plans will be very detailed and several pages long while other action plans will be very short. What is 
important is that the action plan includes all of the steps needed to successfully implement the strategy. 

When identifying responsible parties and timelines, it is important to be specific. Identifiers such as “team” or “staff” 
and vague deadlines such as “ASAP” or “ongoing” eliminate accountability and reduce the chances that the action 
steps will be completed on time. As much as possible, the team should assign responsibilities to specific people with 
specific deadlines. 

Communication is a key component of a successful action plan. If the team chooses to assign responsibilities to 
people who are absent or who are not members of the data team, it is important to secure their consent prior to 
finalizing the assignment and to designate a team member to follow-up to ensure that responsibilities are clearly 
communicated and understood. 
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To streamline monitoring of the action plan, the team may want to appoint a single person to lead it. This person 
would hold responsibility for checking in with responsible parties to learn what progress has been made, ensure that 
deadlines are met, and make certain that the necessary resources are secured. This individual would also be responsible 
for reminding the group about upcoming milestones and deadlines, as well as reporting to the group on progress 
made. 

Implement the Action Plan 

During implementation, the data team will serve as troubleshooter, monitoring the implementation activities and 
intervening as needed (Tool 6B, page 46). Once the plan is implemented, it is important to continually build trust by 
carefully designing conversations among colleagues (Boudett et al., 2007). Building structures and processes for the 
team to routinely talk about the implementation with colleagues allows team members to build collegiality, trust, and 
buy-in among members of the school or district community. 

During implementation, the team should check in regularly, celebrate successes, identify challenges, and make changes 
as needed. If the action plan gets off track, perhaps through dropped responsibilities or unforeseen challenges, the 
team should troubleshoot and determine actions to bring it back on track. 

Evaluate Effectiveness of the Strategy 

Assessing progress is essential in the data inquiry cycle and to assure improvement. Without measuring effectiveness 
of actions, the data team will not know whether any progress has been made or what progress looks like in the 
context of the cycle of inquiry (Boudett et al., 2007). Likewise, once the strategy has been implemented in full, it is 
important for the team to measure whether it has achieved the intended outcome. 

The intended outcome and deadline have already been outlined in the strategic statement. The team identified a 
baseline and strategic goal when it wrote and adopted the strategic statement. For example: 

By May, 2013, improve the 3 rd -grade attendance rate from 78 percent to 90 percent by moving math classes to the 
afternoon. 


In this statement, the baseline is a 78 percent attendance rate, and the goal is a 90 percent attendance rate. This 
notifies the team that they will need to measure 3 rd -grade attendance rates, seeking a change of at least +12 percent. 
The deadline is May 2013, which communicates information on when to measure the effectiveness of the strategy. 

The team may want to determine whether the strategy has been carried out in its entirety or is still underway. In 
the above example, the team can simply ask whether math classes had been successfully moved to the afternoon. 
But ongoing strategies may be less straightforward. For example, if a strategy statement says that math teachers 
will provide 10 extra minutes of independent practice time each week for the duration of the school year, the team 
must check in periodically to determine whether the strategy has been fully implemented and is continuing to be 
implemented. 

In assessing the effectiveness of the strategy, the team should collect data from the same source used to establish 
the baseline. For example, if the team measured the baseline of 3 rd -grade attendance rates from the school’s internal 
database at the end of the prior year, the new attendance rates should also be measured using that same database 
when it comes time to determine progress. 
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It is important to note that the team cannot evaluate the effectiveness of the strategy until the deadline has passed. 

For example, in the previous strategic statement, the team states it cannot know until after May 2013 whether the 
attendance rate improved to 90 percent. Some strategies will be short-term and lend themselves to quick evaluations 
of effectiveness; other strategies will be more long-term and may require a considerable waiting period (such as 
strategies relying on annual standardized test outcomes). The team can simply move on to another cycle of data 
inquiry while it waits to measure the effectiveness of the first strategy. 

Evaluate Progress and Improvement Within Focus Area 

Improvement within the overarching focus area is the ultimate goal of the cycle of data inquiry. For this reason, it’s 
important for the team to check in on the initiative once it has been implemented to see if progress has been made. 
This can be done in the same way the team evaluated the effectiveness of the strategy, but it does not need to be 
completed at the end of every strategy, particularly if the team is pursuing an ongoing cycle of numerous short-term 
action strategies. 

The team should consider how it can measure or assess improvement. For example, a common focus area will be 
“student achievement.” The team might check on common measures of achievement, such as state standardized test 
scores or retention rates, annually, to measure whether the implemented strategies are having a positive impact. 

This evaluation will provide opportunities for team members and their colleagues to observe the results of their 
efforts. When people see positive impacts on student achievement, their motivation and continued effort increases. 

In addition, evaluating the effectiveness of the strategies and actions allows the team to learn what is and is not 
working. When actions or strategies are not working well, the team can make adjustments as needed. 

What if Improvement Isn’t Made? 

Sometimes, evaluating the effectiveness of the strategic actions might indicate they had litde or no effect. Sometimes, 
it might even indicate that a seemingly negative result occurred. While this is understandably frustrating, it’s important 
for the team to recognize that such an outcome provides valuable information by eliminating one or more possibilities 
for driving factors, key challenges, or effective actions. The cycle may have generated increased knowledge or deeper 
questions to drive a future cycle of inquiry. It has also provided the team members with an opportunity to practice 
their data inquiry skills. 

Other times, the measurements may show that some progress has been made but not the desired change. For 
example, in the previous example about 3 rd -grade attendance rates, the team might discover that the rate in May 2013 
was 86 percent, which is an improvement but falls short of the strategic goal of 90 percent. The team may want to 
re-measure the data again at the close of each upcoming semester, to determine whether progress is continuing and 
appears to be on track. 

It’s possible that many strategies will require more time to achieve the desired change than what was specified in the 
initial strategic statement. This is acceptable, and the team can simply move on to a new cycle of data inquiry while 
continuing to monitor the effectiveness of the ongoing strategy. 
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Keep Working! 

Having completed a cycle of data inquiry, the data team is now ready to move on to another one. The new cycle might 
consider the same overarching focus area, the same focus question, or even the same key challenge. What is important 
is that the team follows all six stages in implementing new, data-based strategies and actions. 

School improvement is a continuous process. Once the team has completed its first cycle of inquiry, it should 
celebrate its successes, revisit its initial goals and assumptions, and plan to move forward into the next cycle with 
innovative ideas and excitement (Boudett et al., 2007). The improvement cycle returns to the data-driven decision 
making process, but at a higher level focused on a potentially more complex challenges in order to elevate student 
achievement even higher. 

Eventually, inquiry and data-based decision making will become embedded in the culture of the organization. All 
members of the school or district community should begin to ask probing questions about student achievement using 
various kinds of data to check assumptions, generate dialogue, identify plans for action, and gauge progress. 
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Data Team Logistics 


Building the Data Team 


A data team may go by several names, including Professional Learning Community (PLC), but it is always a group of 
people dedicated to improving student achievement through data inquiry. 

The data team does not hold sole responsibility for all of the work involved in pursuing improvement. It is a group of 
people who have agreed to dedicate time and effort to organize change efforts around data, and who will help build 
and maintain a culture of data use. 

Who is on the Data Team? 

The data team’s composition may depend in large part on the identity, goals, and focus of its organization. Regardless 
of who chooses to take part, practitioners suggest keeping membership to a maximum of 1 5 people, to enable 
effectiveness and efficiency (Learning Point Associates, 2004). 

Data teams may be small grade-level or department groups, made up largely of teachers, with the goal of improving 
student achievement within their grade level or department. They may be school-wide teams, composed of teachers, 
support personnel, and administrators, with a goal of improving student achievement across the entire school 
or at the grade or department level. They may be individual teachers seeking to make improvements within their 
classrooms. They may be district-wide teams, consisting of representatives from schools, grade levels, and different 
professional roles, with the goal of improving student achievement district-wide. Or they may be an entirely different 
grouping altogether. 

Potential data team members should be asked if they are willing and able to make the commitments in time and effort 
that are required to effectively work toward making data-based decisions for change within the organization. Some 
data teams ask members to sign a statement indicating their dedication to the effort and agreement with the team’s 
expectations and norms. 

Confidentiality 

Because the data team is likely to use identifiable student data, it is important to have the team members discuss and 
understand the parameters of confidentiality. Every member should become familiar with the Nebraska Department 
of Education’s Data Access and Use Policy and Procedures document (available at http:/ /www.education. ne.gov/nssrs/ 
docs/Nebraska_Data_Policy_December_2010.pdf) and should complete a confidentiality agreement. A sample 
agreement appears on page 32. All agreements should be approved by an administrator prior to use. 


31 


Sample Confidentiality Agreement 


I acknowledge that as a member of 's data team, I may have access to confidential, 

personally-identifiable information for the purposes of pursuing student achievement and school improvement 
efforts. 

As a member of this data team, I acknowledge that I have received a copy and understand the content of the 
Nebraska Department of Education’s Data Access and Use Policy and Procedures document. 

I understand that, under the Family Educational Rights and Privacy Act (FERPA) and the State of Nebraska’s data 
access and use policy and procedures, it is my professional responsibility to maintain the confidentiality of all 
student data and information. I understand that this means that I: 

• Cannot use, reveal, or in any other manner disclose any personally-identifiable information furnished, acquired, 
retrieved, derived, or assembled by me or others for any purpose other than the projects undertaken by the 
data team 

• Must comply with the Family Educational Rights and Privacy Act (FERPA) [20 U.S.C. 1232g; 34 CFR Part 99] 

I pledge to adhere to all data confidentiality guidelines applicable to my work as a member of the data team. 
Signature: 


Printed Name: 


School or District: 


Date: / / 


Rosters and Attendance 

The team should track members on a roster (below), to ensure that everyone on the data team knows who is on the 
team and how to reach them. A blank copy of the roster can be repurposed as a simple sign-in sheet for each data 
team meeting, to facilitate tracking of membership and attendance. 

Data Team Roster 


Team Member Name 

Role in Organization 

E-mail/Phone 

Role on Team 
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Data Team Roles and Responsibilities 

Members of the data team should be assigned roles to ensure full commitment and equal participation among 

members. These roles can rotate, so that every member holds every role at some point, or they can be static. Members 

can hold more than one role at a time. Important and common roles include: 

• Facilitator: Guides the team through the stages of data inquiry. When decisions need to be made, this person will 
lead the team through decision making and consensus building. If the team does not appoint referee or planner 
roles, the facilitator will also maintain responsibility for the tasks associated with those roles. 

• Time-keeper: Responsible for keeping the meeting moving according to the schedule by beginning and ending on 
time. 

• Recorder: Responsible for taking notes during the meeting and reporting and disseminating to all team members 
following the meeting. 

• Referee: Responsible for keeping the team on task during the meeting. 

• Planner: Responsible for communicating with members about upcoming meetings and responsibilities. 

• Active Participant: All team members will be engaged participants in the team meetings, actively listening and 
contributing to discussions. 

Each data team should decide which roles are needed to accomplish the work. Other roles may be necessary in 

addition to those outlined above. 

Establishing Norms and Purpose 


High-functioning data teams appreciate the challenges inherent in the work they undertake and the investments 
required of all members, in terms of time and effort. These teams work hard to build a strong foundation and to 
identify shared norms and statements of purpose (Love, Haley-Speca, & Reed, 2010). 

In early meetings, the team should come to consensus on the group’s shared values and expectations for behavior. 
They should develop a clear and shared understanding of the team’s purpose, specific roles to be filled, responsibilities 
of each member, and how the team will conduct its meetings (Love, Haley-Speca, & Reed, 2010). 

One method of creating this shared understanding is group brainstorming at an early meeting. This can be conducted 
on paper (see Identify Values and Behavioral Norms on page 34) or by giving participants sticky notes on which they 
can write their ideas and post them to a large piece of paper. 

Norms 

Ask the group members to consider their previous team-based experiences. What did they like about those teams? 
What frustrated them about those teams? The answers can help shape expectations about behavior. 

• Positive Behaviors: What behaviors do we want our group members to exhibit? 

Examples: Active listening, respect for each member’s contributions, following through on promises made 

• Negative Behaviors: What behaviors are unacceptable within the group? 

Examples: Arriving to meetings late, texting or e-mailing during the meeting, making excuses for low student 
achievement 
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From these inputs, the group can create a list of group norms. This list should be compiled from everyone’s 
contributions and reviewed and approved by all members before it is finalized. The group should review the norms 
at the beginning of every meeting. Some teams like to include them on each meeting’s agenda to ensure ongoing 
understanding and awareness. 

Values and Purpose 

Group members should also consider the purpose of the team. If the team was convened by an administrator, it is 
important for team members to understand the reasons for the team’s creation and what the administrator hopes the 
team will accomplish. 

Team members might ask themselves why they have joined the data team and what they hope to accomplish. From 
these thoughts, consider: 

• What do we believe? 

• What are we here to do? 

From this brainstorming, the group can create a belief statement and a statement of purpose. These should be 
created from the contributions of the team and reviewed and approved by all members before they are finalized. The 
group should include the statements of belief and purpose in all agendas, to ensure ongoing shared understanding. 

Identify Values and Behavioral Norms 


Values & Purpose 

Behavioral Norms 

What do we believe? 

What are we here to do? 

Positive: What behaviors do we want to exhibit? 

Negative: What behaviors are unacceptable? 
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Meeting Organization 

To keep the team on task and moving forward, every meeting should have an agenda to guide the discussion. 
Important items to include on the agenda are: 

• Affirmation of team norms and values 

• A review of the status of the last meeting’s action items 

• Objective and expected outcomes for the meeting 

• Action items to pursue for the next meeting (including identifying who is responsible) 

• Time allotments for each agenda item 

The purpose of the agenda is to ensure that all necessary items are covered and that the team members’ time is used 
efficiently and effectively. All agendas and notes should be maintained in a central repository, such as a computer file, 
for later review. 

Sample Agenda 


ABC School Data Team Meeting 


Meeting Date and Time 

February 29, 2012, 3:00- 4:00 pm 

Meeting Objective 

Complete Stage 3 of the Data Guide: SURFing the Data 

Expected Outcome(s) 

• Completed SURF statements related to our focus question 

• Action items for next meeting 

Team Norms 

1 . We will arrive to our meetings on time, end on time, and remain engaged and 
focused throughout each meeting. 

2. We will not presuppose reasons for anything but will base all decisions on what 
the data tell us. 

3. We will listen to one another respectfully and will thoughtfully consider one 
another’s opinions. 

Statement of Belief 
and Purpose 

We believe that all students can achieve at a high level and that there are no 
excuses for low achievement. We exist to identify solutions for even the toughest 
challenges to ensure that all students can achieve at a high level at ABC School. 


3 : 00 - 3:05 

Review Norms & Beliefs 


3 : 05 - 3:15 

Review Action Items from 1/25 meeting 


3 : 15 - 3:30 

SURF Statements 

Divide into three small groups. Each 
group reviews one of the three data 
sources. Write out SURF statements on 

easels. 

3 : 30 - 3:45 

Readout 

Readout each team’s SURF statements. 

3 : 45 - 3:55 

Action Items 

Identify action items for next meeting. 

3 : 55 - 4:00 

. . j.. Recognize team members for 

Appreciations & Closing 

successes. 
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Data Inquiry Toolkit 


Overview 


Once the infrastructure is settled, the data team members will begin its work by following the stages of the data 
inquiry process. Each stage is discussed in detail in the Background Information section and has associated tools 
to assist the team members in completing the stages. The stages and their corresponding Background Information 
sections are listed below. 


Stage 1: 

Ask Good Questions 

Corresponds to Background Information pages 8—10 

Stage 2: 

Collect and Prepare the Data 

Corresponds to Background Information pages 11—14 

Stage 3: 

Find Trends and Make Observations 

Corresponds to Background Information pages 15-19 

Stage 4: 

Interpret the Data 

Corresponds to Background Information pages 19-22 

Stage 5: 

Plan for Action 

Corresponds to Background Information pages 23-26 

Stage 6: 

Implement, Monitor, and Sustain 

Corresponds to Background Information pages 27—30 
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Stage 1 : Ask Good Questions 


1A. Identify the Focus Area 


FOCUS AREA FOR THE CYCLE OF DATA INQUIRY: 


IB. Generate Possible Questions 


Question 

1 


2 


3 


4 


5 


6 


7 


8 



1C. Narrow the Question to Focus Inquiry 


Top Priority Question: 

1 . Does the question address an issue that is 
significant to the individual, group, or the 
school? 

□ Yes 

□ No 

If no, refine the question or choose a different 
question that will result in more significant new 
insight. 

2. Is it a question with a dichotomous answer (e . g . , 
yes/no or improving/not improving)? 

□ Yes 

□ No 

If yes, refine the question to be one with a 
more informative answer or choose a different 
question. 

3. Is the question amenable to action? Can you act 
on the outcomes in order to make things better? 

□ Yes 

□ No 

If no, refine the question or choose a different 
question that will result in more actionable 
information. 

4. Is it possible to address the question in the 
available time? 

□ Yes 

□ No 

If no, narrow the scope of the question to be 
more manageable in the time allotted for this 
cycle. 

5. Is the question narrow, clear, and 
straightforward? 

□ Yes 

□ No 

If no, narrow the scope or refine the question to 
clarify. 

6. Are the data necessary for answering the 
question accessible prior to the time of inquiry? 

□ Yes 

□ No 

If no, either modify the question to one that can 
be answered with available data or set the date 
of study when necessary data can be collected. 

Final Focus Question: 
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Stage 2: Collect and Prepare the Data 


2A. Data Collection Plan 


Focus Question: 

What data are 
currently available 
that will help 
answer the focus 
question? 

What factors 
will be used for 
disaggregation? 

How will the data 
be prepared and 
organized? 

Who is responsible 
for collecting and 
organizing each 
data source for the 
team? 

Deadline for data 
to be collected and 
prepared 

EXAMPLE 

• 3 rd -grade NeSA 
scores for 2010 
and 2011 

• Gender, race/ 
ethnicity, and 
classroom teacher 

• One table for each 
year. The columns 
will be NeSA score, 
gender, ethnicity, 
and teacher. The 
rows will be student 

IDs. 

• Ralph and Sheri 

• Next meeting: 
3/14/12 
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Stage 3: Find Trends and Make Observations 


3A. Data Review 


Focus Question: 

Data Source: 

Check to ensure the statements are: 

s Specific s Understandable s Related to the focus question s Factual 

Statements about the data source 
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Stage 4: Interpret the Data 


4A. Celebrate Strengths and Identify Challenges 


Observed Strengths 

Observed Challenges 




4B. Prioritize Challenges 


Priority 

Challenge 

1 


2 


3 


4 


5 
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4C. Determine Driving Factors 



4D. Cause and Effect/Fishbone Diagram 
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4Di. Fishbone Diagram: Why/Because Table 


Key Challenge: 

Why? 

Because... (Driving Factors) 

Which Area? 



Concern Influence Control 


^ 

Concern Influence Control 

* 


Concern Influence Control 

* 




4E. Isolate Controllable Driving Factors 


Controllable or High-Influence Driving Factors 

Impact 

Ranking 

















Top Driving Factor: 


Impact Rankings: 

1 = No impact on the key challenge 
3 = Some impact on the key challenge 
5 = High impact on the key challenge 
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Stage 5: Plan for Action 


5A. Identify Strategy for Action 


Focus Area: 

Focus Question: 

Key Challenge: 

Top Driving Factor: 

Strategic Target: 

Potential Strategies 

Priority Ranking 

Is It Possible Given: 

Time? 

Resources? 





























Strategy for Action: 


5B. Strategy Setting Tool 


Strategic Statement: 



Ry . improve 


from 

(date) 

(key challenge) 

(current data point) 

to by 



(future data point) 

(strategy) 
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Stage 6: Implement, Monitor, and Sustain 


6A. Action Plan - Organizing the Team for Action 


Strategic Statement: 

Action Steps 

What will be done? 

Resources 

What is needed to do this 
step? 

Deadline 

By when? 

Responsibilities 

Who will do it? 
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6B. Evaluating Progress and Improvement 


Focus Area: 

Key Challenge: 

Strategic Statement: 

Date 

Baseline 

Measurement 

New 

Measurement 

Difference 

Has Desired Change Been 
Achieved? 

Next Steps 

Key Challenge 

Focus Area 





Y N 

Y N 






Y N 

Y N 






Y N 

Y N 






Y N 

Y N 






Y N 

Y N 


Has the team achieved 
improvement in the focus 
area? 

If yes, describe: 
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Organizational Tools 


7A. Parking Lot Tool 


Questions to Return to Later 

Data Findings to Return to Later 




7B. Data Cycle Overview 


Focus Area 


Focus Question 


Key Challenge 


Top Driving Factor 


Strategy 


Strategic 

Statement 
(including Target) 
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Sample Data 


Background 


Mr. Russell, the principal at Triumph Middle School in Nebraska, has been concerned about his students’ attendance 
rates and NeSA scores. He isn’t sure, but he thinks that his school’s scores are trending downward. The principal 
wants to convene a group to investigate ways to improve students’ achievement, particularly on the NeSA assessment. 

He begins by announcing at a staff meeting that this issue is at the top of his school improvement agenda and that he 
plans to assemble a group of teachers to consider how Triumph can improve scores over the 2012—2013 school year. 
He asks for volunteers, and four teachers and a guidance counselor sign up and become the data team for this project. 

The team begins to meet regularly to conduct a cycle of data inquiry around Mr. Russell’s charge of improving 
student achievement school-wide. The following is an outline of their cycle and the results they achieved. 


Stage 1 : Ask Good Questions 


1A. Identify the Focus Area 

The focus area for inquiry was identified by Mr. Russell and served as the impetus for forming the data team. The 
team restates this focus area to ensure full group understanding. The facilitator hangs a piece of chart paper on the 
wall with the full data cycle overview copied on it (tool 7B) . She adds the focus area to the top row. 

FOCUS AREA FOR THE CYCLE OF DATA INQUIRY 
Improve student achievement, particularly as measured by the NESA test. 


The Triumph data team now knows that its work will be focused on improving student achievement on the NeSA 
assessments. This will help them to narrow their questions for inquiry. 

The data team’s facilitator begins the questioning process by reminding team members to concentrate on the focus 
area. She also reminds the team that, though only one question will be used for this cycle of data inquiry, any other 
questions will remain available for subsequent or concurrent cycles of inquiry. She sets 20 minutes for brainstorming, 
after which the team must choose a question for further refinement and inquiry. 

As the team suggests questions for inquiry, the data team’s recorder documents them in the focused brainstorming 
spreadsheet tool (IB, page 38). The facilitator redirects team members when necessary and ensures that all of the 
questions are reasonably focused and within the realm of reading achievement. 
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IB. Generate Possible Questions 


Question 

Prefer? 

1 

Which grades show the most improvement on NeSA and why? 

Yes, No 

2 

Is there a correlation between math scores and reading scores? 

No 

3 

Do attendance rates have any effect on NeSA test outcomes? 

Yes 

4 

Why don’t students understand that NeSA is important? 

No 

5 

Which teachers are producing the best outcomes and what are they doing differently in 
their classrooms? 

No 


At the end of the 20 minutes, the recorder reads out the five questions that the group has identified. Next, he uses the 
forced-choice method to help the group choose its top priority question. Instead of prioritization rankings, he enters 
Yes/No markers in the far right column of the tool. The recorder then leads the group through the forced choices by 
asking them to consider which questions will have the greatest impact on the focus area. 

Recorder: “Which do you prefer, Question 1 or Question 2?” 

The team votes by holding up one finger for Question 1 and two fingers for Question 2. Question 1 is chosen. 

Recorder: “Which do you prefer, Question 1 or Question 3?” 

Question 3 is chosen. This continues until the team has settled on Question 3 as its focus question. 

1C. Narrow the Question to Focus Inquiry 

The team now considers whether or not the question they chose is sufficiently narrow, clear, and focused for effective 
inquiry. The chosen question is dichotomous— it can be answered as ‘Yes, attendance rates have an effect,” or “No, 
they do not.” The team works to refine this question to something more informative: What effects, if any, do attendance 
rates have on NeSA test outcomes'? They go through the questions again and determine that this is a stronger question. 
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Top Priority Question: 

1 . Does the question address an issue that is 
significant to the individual, group, or the school? 

s Yes 

□ No 

If no, refine the question or choose a different 
question that will result in more significant new 
insight. 

2. Is it a question with a dichotomous answer (e.g., 
yes/no or improving/not improving)? 

Y Yes 

□ No 

If yes, refine the question to be one with a 
more informative answer or choose a different 
question. 

3. Is the question amenable to action? Can you act on 
the outcomes in order to make things better? 

■/ Yes 

□ No 

If no, refine the question or choose a different 
question that will result in more actionable 
information. 

4. Is it possible to address the question in the 
available time? 

S Yes 

□ No 

If no, narrow the scope of the question to be 
more manageable in the time allotted for this 
cycle. 

5. Is the question narrow, clear, and straightforward? 

S Yes 

□ No 

If no, narrow the scope or refine the question to 
clarify. 

6. Are the data necessary for answering the question 
accessible prior to the time of inquiry? 

S Yes 

□ No 

If no, either modify the question to one that can 
be answered with available data or set the date 
of study when necessary data can be collected. 

FINAL FOCUS QUESTION: 

What effects, if any, do attendance rates have on NeSA test outcomes? 


The facilitator copies the final focus question to the data cycle overview chart paper. The team briefly celebrates the 
completion of Stage 1 before moving on to Stage 2. 


Stage 2: Collect and Prepare the Data 


While discussing their data collection plan, the team becomes concerned with the number of data points they will be 
collecting if they choose to analyze this question for all students in grades 6—8. To make their cycle more manageable, 
the team decides to focus this first cycle of data inquiry on 7 th -grade students only. 

2A. Data Collection Plan 


Focus Question: What effects, if any, do attendance rates have on NESA test outcomes for 7 ,h -grade students? 

What data are currently 
available that will 
help answer the focus 
question? 

What factors 
will be used for 
disaggregation? 

How will the 
data be prepared 
and organized? 

Who is responsible 
for collecting and 
organizing each data 
source for the team? 

Deadline for 
data to be 
collected and 
prepared 

The scores from the most 
recent NESA-M test for 7 th 
graders (Spring 201 1) 

Gender 

Classroom teacher 

In a single table, 
with students 
in rows and 
their scores and 
demographics in 
columns 

Meredith and Jeremy 

Next meeting: 

9/12/11 

The scores from the most 
recent NESA-R test for 7 th 
graders (Spring 201 1) 

Average attendance rates 
for the first three quarters 
of 201 1 for 7 th graders 
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Meredith and Jeremy volunteer to work together to collect and prepare the data because they both have access to the 
databases where this information is kept. 

2B. Prepare the Data 

In preparing the data, Meredith and Jeremy color code all of the scores and levels and sort the data by student ID. 
They make sure to provide a legend outlining what the different color codes represent so that it will be easy for the 
team to read. In general, red indicates low scores or levels and green indicates high scores or levels. 


Student ID 

Gender 

Teacher 

Attendance 

Rate 

NeSA-M 

NeSA-R 

101617 

F 

A. Cross 

65% 

Below 

Below 

121224 

M 

L. Grimes 

95% 

Exceeds 

Meets 

144084 

F 

L. Grimes 

100% 

Exceeds 

Exceeds 

230769 

M 

A. Cross 

82% 

Meets 

Meets 

257661 

F 

L. Grimes 

82% 

Meets 

Meets 

353042 

F 

D. Berber 

90% 

Below 

Meets 

385537 

M 

L. Grimes 

95% 

Below 

Exceeds 

527890 

M 

D. Berber 

49% 

Below 

Below 

623058 

F 

L. Grimes 

100% 

Exceeds 

Exceeds 

714597 

M 

L. Grimes 

48% 

Meets 

Below 

720508 

F 

D. Berber 

92% 

Meets 

Exceeds 

747337 

M 

D. Berber 

69% 

Below 

Below 

762871 

F 

A. Cross 

90% 

Meets 

Meets 

829757 

M 

A. Cross 

96% 

Meets 

Exceeds 

864729 

F 

D. Berber 

89% 

Exceeds 

Meets 

905777 

M 

A. Cross 

84% 

Meets 

Meets 

910519 

F 

L. Grimes 

85% 

Below 

Exceeds 

926985 

M 

A. Cross 

50% 

Below 

Meets 

979696 

F 

D. Berber 

70% 

Below 

Below 

980957 

F 

D. Berber 

50% 

Below 

Below 

993067 

M 

A. Cross 

58% 

Below 

Meets 


Attendance Rates 

NESA Scores 

0%-50% 

Poor 

Below 

Red 

51 %-85% 

Fair 

Meets 

Yellow 

86%-1 00% 

Good 

Exceeds 

Green 
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Noticing that there appears to be a difference in attendance and scores across classrooms, Meredith and Jeremy also 
do some quick calculations to determine the average attendance rates for each teacher. They create a graph to display 
the attendance rates by classroom. 

Meredith and Jeremy send the table and the graph out to the team a day ahead of the next meeting to give everyone a 
chance to review it to ensure an efficient and effective meeting. They also save all of the data tables to the computer 
they will use during the meeting. 

Average Attendance Rates, by Teacher 



Stage 3: Find Trends and Make Observations 


At the next meeting, the team arrives with excitement, having reviewed the data set before the meeting. The facilitator 
has brought chart paper and markers, and the recorder has prepared a SURFing table to document all of the ideas that 
are raised. Meredith has volunteered to re-sort and conduct calculations on the data as requested by the team during 
the meeting, projecting it onto a screen so everyone can see. To ensure that their meeting ends on time, the facilitator 
sets a five-minute time limit for making observations on each data set and 1 5 minutes for the combined data sets. 

The team begins by reviewing the principles of SURFing (especially the part requiring the statements to be unbiased) 
and then takes turns providing observations, which are recorded. When additional questions or ideas for further 
inquiry are raised during the SURFing process, the facilitator notes them on the chart paper, which serves as the 
team’s parking lot of ideas for this cycle of inquiry. 

The team reviews the NeSA results, followed by the attendance rates. While working through the data sets, team 
members often ask Meredith to make some calculations. For example, they determine that the average attendance rate 
was 78 percent, and they count 10 students who scored “Below” in the NeSA-M. 
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3A. SURFing the Data 


Focus Question: What effects, if any, do attendance rates have on NeSA test outcomes for 7 ,h -grade students? 

Data Source: Spring 201 1 NeSA-R scores for 7 th graders 

Check to ensure the statements are: 

s Specific s Understandable s Related to the focus question s Factual 

Statements about the data source 

• 6 students scored below 

• 6 students exceeded 

• 9 students met 

Data Source: Spring 201 1 NeSA-M scores for 7 th graders 

Statements about the data source 

• 10 students scored below 

• 4 students exceeded 

• 7 students met 

Data Source: 2010-2011 Attendance rates up to NeSA tests, 7 th graders 

• 4 students have poor attendance rates 

• 6 students have fair attendance rates 

• 1 1 students have good attendance rates 

• 2 students have perfect attendance 


After the time limit is reached for each data set, they move on to the next. After reviewing all three data sets 
separately, they consider the full table, with all of the data sets combined. They ask Meredith to re-sort the data on 
each disaggregation variable in order to better determine how the data look when analyzing across subgroups. 

For example, when they ask Meredith to re-sort the data according to gender, they see the following table, where all 
of the girls are grouped together at the top and all of the boys are grouped together at the bottom. This re-sort allows 
them to quickly see that most of the poor attenders were boys. They can also see that there did not appear to be a 
large difference across genders in the number of students who scored “Below” on the NeSA-M. 
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Student ID 

Gender 

Teacher 

Attendance 

Rate 

NeSA-M 

NeSA-R 

101617 

F 

A. Cross 

65% 

Below 

Below 

762871 

F 

A. Cross 

90% 

Meets 

Meets 

353042 

F 

D. Berber 

90% 

Below 

Meets 

720508 

F 

D. Berber 

92% 

Meets 

Exceeds 

864729 

F 

D. Berber 

89% 

Exceeds 

Meets 

979696 

F 

D. Berber 

70% 

Below 

Below 

980957 

F 

D. Berber 

50% 

Below 

Below 

144084 

F 

L. Grimes 

1 00% 

Exceeds 

Exceeds 

257661 

F 

L. Grimes 

82% 

Meets 

Meets 

623058 

F 

L. Grimes 

1 00% 

Exceeds 

Exceeds 

910519 

F 

L. Grimes 

85% 

Below 

Exceeds 

230769 

M 

A. Cross 

82% 

Meets 

Meets 

829757 

M 

A. Cross 

96% 

Meets 

Exceeds 

905777 

M 

A. Cross 

84% 

Meets 

Meets 

926985 

M 

A. Cross 

50% 

Below 

Meets 

993067 

M 

A. Cross 

58% 

Below 

Meets 

527890 

M 

D. Berber 

49% 

Below 

Below 

747337 

M 

D. Berber 

69% 

Below 

Below 

121224 

M 

L. Grimes 

95% 

Exceeds 

Meets 

385537 

M 

L. Grimes 

95% 

Below 

Exceeds 

714597 

M 

L. Grimes 

48% 

Meets 

Below 


Continuing with the sorting and calculations, the team spends 15 minutes identifying factual relationships between the 
data sources while the recorder notes them on the SURF tool. 
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Focus Question: What effects, if any, do attendance rates have on NeSA test outcomes for 7 ,h -grade students? 

Data Source: All 3 of the data sources (NeSA-R, NeSA-M, attendance rates) plus demographics 

Check to ensure the statements are: 

s Specific v' Understandable s Related to the focus question s Factual 

Statements about the data source 

• 3 of the 4 students with poor attendance scored below in NeSA-M 

• The student with the lowest attendance rate (48%) met standards In NeSA-M 

• 4 of the students with the lowest rates in the fair attendance category scored below in NeSA-M 

• 2 of the 9 students with good attendance scored below in NeSA-M 

• 4 of the 9 students with good attendance exceeded in NeSA-M, 3 met in NeSA-M 

• The 3 students with the lowest attendance rates scored below in NeSA-R 

• Only 1 student with poor or fair attendance (n=1 2) exceeded in NeSA-R 

• None of the students with good attendance scored below in NeSA-R\ 

• 5 of the 9 students with good attendance exceeded in NeSA-R 

Demographics 

• 1 female and 3 male students had poor attendance 

• The 2 students with perfect attendance were both female and both in Grimes’ class 

• 5 students of each gender scored below on NeSA-M 

• 3 girls and 1 boy exceeded in NeSA-M 

• 3 students of each gender scored below on NeSA-R 

• 4 girls and 2 boys exceeded on NeSA-R 

• Grimes had the best attendance and Berber the worst with a difference of 13%: 

Grimes 86% 

Cross 75% 

Berber 73% 

• Berber’s class had the lowest performance on NeSA-M: 5 out of 7 below, 1 exceeds, 1 meets 

• Berber’s class also had the lowest performance on NeSA-R: 4 out of 7 below, 1 exceeds, 2 meets 

• Grimes’ class had the highest performance on each test: 

NeSA-M 3 exceeds, 2 meets (out of 7) 

NeSA-R 4 exceeds, 2 meets (out of 7) 
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Stage 4: Interpret the Data 


Now that the team has identified factual statements evident in the data, they are excited to begin interpreting those 
statements, as they move forward in identifying potential answers to their initial question. First, they consider what the 
data demonstrate about success in achievement and attendance. Next, they consider what the data demonstrate about 
challenges in achievement and attendance. 


4A. Celebrate Strengths and Identify Challenges 


Observed Strengths 

Observed Challenges 

• Two students had perfect attendance. 

• The 7 th grade had an average attendance rate of 78%. 

• The majority of students were at or above expectations 
in both NeSA-R and NeSA-M. 

• Students with poor attendance seem to be achieving at 
a lower rate than students with good attendance. 

• Grimes’ students had high scores, with 5/7 meeting or 
exceeding in NeSA-M and 6/7 meeting or exceeding in 
NeSA-R. 

• Cross and Berber have average classroom attendance 
rates below the grade average (75% and 73%, 
respectively). 


• Girls appear to be achieving at a slightly higher rate and 
have attendance rates 10 percentage points higher than 
boys’ attendance rates. 


The team decides to share the successes with the greater school community at the next all-school meeting. As a group, 
they rank order the challenges, with number 1 being the top priority challenge. This is the challenge that the team 
thinks will have the most impact in answering the focus question. 

4B. Prioritize Challenges 


Priority 

Challege 

1 

Students with poor attendance seem to be achieving at a lower rate than students with good 
attendance. 

2 

The 7 th grade had an average attendance rate of 78%. 

3 

Cross and Berber have average classroom attendance rate below the grade average (75% and 73%, 
respectively). 

4 

Girls seem to be achieving at a slightly higher rate and have attendance rates 10 percentage points 
higher than boys’ attendance rates. 


The team decides that the apparent connection between attendance and NeSA scores is the top challenge that also 
relates to their focus question. At this point, they can choose to use either the Determine Driving Factors tool (4C, 
page 42) or the Fishbone Diagram (4D, page 42). Because they want to delve deeply into the potential root causes of 
this correlation, they decide to use the Fishbone Diagram method of identifying driving factors. The facilitator draws 
a large blank diagram on the chalkboard in the meeting room. 
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They start by placing the key challenge into the “head” of the diagram. Next, they take five minutes to work as a 
team to identify major categories of causes for the key challenge (“Transportation,” “Motivation,” “Physical Time at 
School,” and “Parental Oversight and Guidance”). The facilitator distributes sticky notes and pens and asks the team 
members to take five minutes to generate possible causes within each category. The team members write each cause 
on a separate sticky note and affix them to the diagram below the appropriate category. After the five minutes has 
concluded, the team has identified eleven potential causes related to the key challenge. 

4D. Fishbone Diagram 


Transportation 


Motivation 



Students with poor 
attendance are 
achieving at a lower 
rate than students 
with good attendance. 


Next, the team works through the potential causes to arrive at driving factors of those causes. Because they have only 
twenty minutes left in their meeting, they isolate two of the causes that they agree are most actionable and have the 
most impact on the key challenge — “Transportation: Can’t afford bus passes” and “Physical Time at School: Skipping 
school.” They complete two Fishbone Diagrams: Why/Because Tables, one for each cause, to arrive at driving factors. 
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4Di. Fishbone Diagram: Why/Because Table 


Key Challenge: Students with poor attendance seem to be achieving at a lower rate than students with good 
attendance 

Why? 

Because... (Driving Factors) 

Which Area? 

Why can’t they afford bus passes? 

They don’t have any money. 

(Concern) Influence Control 

Why don’t they have any money? 

They can't get jobs. 

((Concern) Influence Control 

Why can’t they get jobs? 

They aren’t old enough yet to work. 

(Concern) Influence Control 


The team realizes it can’t directly control the students’ access to income at this time. While the team could consider 
projects that would directly affect students’ access to bus passes, perhaps by acquiring grant money to pay for them, it 
does not have enough time or resources to work on this driving factor at this time. Team members decide to move on 
to a Why/Because Table for the second potential cause. 


Key Challenge: Students with poor attendance seem to be achieving at a lower rate than students with good 
attendance 

Why? 

Because... (Driving Factors) 

Which Area? 

Why are they skipping school? 

They are able to. 

(Concern) Influence Control 

Why are they able to skip school? 

Their parents don't know about it. 

Concern (Influence) Control 

Why don’t their parents know about it? 

We don’t alert them in a timely 
manner. 

Concern Influence ((ControP) 


The team has identified a driving factor for their key challenge where they can create positive change. They are 
concerned that students aren’t in school often enough to really learn the content. They can influence their skipping 
behaviors by hosting information sessions for parents. They can directly control the parents’ knowledge about the 
students’ attendance patterns by providing them with timely information. 

The team completes this process a few more times until it has a set of potential driving factors. All of the factors that 
are within the area of control are copied to the Isolate Controllable Driving Factors tool (4E, page 43). 
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4E. Isolate Controllable Driving Factors 


Controllable or High-Influence Driving Factors 

Impact 

Ranking 

Students don’t know about the transportation options available to help them get to school. 

3 

The students’ parents don't know they’re skipping school. 

1 

The schedule provides the students with too much free time. 

2 

Top Driving factor: 


The team ranks the potential driving factors by the influence the team members believe the factor has on the key 
challenge. Using the forced-choice method, they identify the second driving factor as their top driving factor. 

They write the key challenge and the top driving factor on the data cycle overview chart to ensure full group 
understanding. The overview is on chart paper in the room so the group members can always see the organizational 
factors of their data cycle process. 

7B. Data Cycle Overview 


Focus Area 

Improve student achievement, particularly as measured by the NeSA assessment. 

Focus Question 

What effects, if any, do attendance rates have on NeSA test outcomes for 7 ,h -grade students? 

Key Challenge 

Students with poor attendance seem to be achieving at a lower rate than students with good 
attendance. 

Top Driving Factor 

Parents are not aware that their children are skipping school. 


Stage 5: Plan for Action 


Now that the team has identified the top driving factor, it is ready to prepare for action. This is an exciting part of the 
process because team members know they will be able to take steps to create positive change for their students. 

In the first step of planning to take action, the team identifies strategies that might positively change the key challenge. 
The facilitator sets a 10-minute time limit for the team to brainstorm possible strategies for action. Unlike a traditional 
brainstorming session, the facilitator asks the team to be very mindful of the relevance of each strategy and allows 
the team to veto irrelevant or truly impractical ideas. As the team collects relevant, feasible ideas for strategies, the 
recorder notes them on a large piece of chart paper. 

In the time allotted, the team identifies numerous potential strategies. While brainstorming, team members veto ideas 
such as “Pass a stronger state law that penalizes parents for students’ non-attendance” and “Provide healthier food in 
the cafeteria” because they are unfeasible and irrelevant, respectively. 
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The team settles on six potential strategies for action and considers whether each is possible given time and resources. 
For example, while the school has the resources, there is no time to update school policies even though team 
members would like to see progress in this area within the next school year; and, while there is enough time, there 
are no resources for hiring school monitors. In addition, though the team likes the idea of a reward system, there is 
no money to fund one. Hoping instead to raise the funds over the coming school year, the team decides to mark the 
“Resources” box as “Not Yet” and to return to this strategy at a later date. 

This leaves three strategies to rank. After some discussion, the team decides that weekly notices to parents will have 
the most impact on the challenge and the least impact on teachers’ time. For the first semester, notices will go out only 
when students have unexcused absences. This will reduce the burden on front office staff and teachers who will need 
to create and track the notices. The recorder adds this strategy to the team’s cycle tracking sheet. 

5A. Identify Strategy for Action 


Focus Area: 

Focus Question: 

Key Challenge: 

Top Driving Factor: 

Strategic Target: Increase 7th-grade average attendance to 90%. 

Potential Strategies 

Priority 

Is It Possible Given: 

Ranking 

Time? 

Resources? 

Hold a parent information night to tell parents about the importance of 
school attendance. 

3 

Yes 

Yes 

Update school policies to make penalties for skipping school more 
severe. 

- 

No 

Yes 

Hire school monitors to patrol the school and grounds to stop 
students from skipping school. 

- 

Yes 

No 

Have teachers call parents when the student is not in class. 

2 

Yes 

yes 

Send home weekly attendance reports that require a parent’s 
signature. 

1 

yes 

Yes 

Create a relevant quarterly reward system, like a pizza party, for 
students who are good attenders. 

- 

Yes 

Not Yet 

Strategy for Action: 




Send home weekly attendance reports that require a parent’s signature whenever a student has an unexcused 
absence. 
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The team knows it will need to have a way to communicate the strategy and its goal to other members of the school 
community. It will also need to know what to measure later to determine if the strategy has been successful. To 
accomplish this, team members write a clear and succinct strategic statement. 

5B. Strategy Setting Tool 


Strategic Statement: 

By the end of this school year, improve 7 ,h -graders’ overall average attendance from 78% to 90% by sending home 
weekly attendance reports that require a parent’s signature whenever a student has an unexcused absence. 


With this strategic statement, the team knows its target (90% attendance rate in the 7* grade by the end of the school 
year) and the plan to reach that target (by sending home attendance reports). This also tells team members what they 
will need to measure at the end of the school year to determine if their strategy has been successful (7 th -grade average 
attendance) . 

Stage 6: Implement, Monitor, and Sustain 


The strategic statement written and agreed upon in Stage 5 drives the action steps in Stage 6. Now, the team outlines 
the specific steps it will take as well as the resources needed, the people responsible, and the deadline for each step. As 
the team puts the plan into action, they track progress closely in order to determine if they’ve met their goals and to 
inform the next cycle of inquiry. 

6A. Action Plan - Organizing the Team for Action 


Strategic Statement: 

By the end of this school year, improve 7 th graders’ overall average attendance from 78% to 90% by sending home 
weekly attendance reports that require a parent’s signature whenever a student has an unexcused absence. 

Action Steps 

What will be done? 

Resources 

What is needed to do 
this step? 

Deadline 

By when? 

Responsibilities 

Who will do it? 

Send letters home with students to inform 
parents about the new process. 

Assistance with 
printing and mailing 
(team will help) 

10/25/11 

Mr. Russell 

Develop unexcused absence reports that can 
be quickly pulled each week. 

None 

11/15/11 

Mrs. Bennett (district 
tech team) 

Develop an automated system for the reports 
to feed into form letters. 

None 

12/1/11 

Ms. Bratton (district 
tech team) 

Mr. Tressel 

Mid-day every Friday, pull the form letters, 
print them, stuff them in envelopes, and mail 
them. 

Envelopes & postage. 
Team will help with 
process in the early 
months. 

Beginning 1/13/12 
and every week 
thereafter 

Mrs. Jefferson & 

Mrs. Moon (front 
office) 

Track the returned, signed letters every 

Monday. Call home if they are not returned by 
Tuesday. 

None 

Beginning 1/16/12 
and every week 
thereafter 

All teachers & front 
office staff 
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Because the team assigned responsibilities to so many 
colleagues who are not part of the data team, it assigns follow- 
up responsibilities to ensure that all individuals with assigned 
responsibilities are aware of and agree to them, and believe they 
have the resources to complete their tasks. During this follow- 
up communication, Jeremy learns from Mr. Tressel that he has 
just been assigned a high-priority district task but that a member 
of his team has agreed to develop the automated system to feed 
the absence reports into form letters. With this information, the 
team updates the action plan. Meredith agrees to track the action 
plan over the first six months (October through March) to ensure 
ongoing completion and compliance. The team agrees to check 
in on the action plan’s progress monthly while completing a new 
cycle of data inquiry. For this cycle, team members decide to 
pursue an amended version of one of the other questions they 
had developed during Stage 1, asking what the relationship might 
be between students’ perceptions of the importance of the NeSA 
tests and their ultimate scores on those tests. 

The data team continues to meet every other week throughout the 
school year. During this time, it completes two more cycles of data 
inquiry, implementing strategies to improve student achievement. 

When June arrives and the school year ends, the data team 
revisits its strategies and begins the process of measuring 
outcomes. For the first cycle of data inquiry, the team pursued 
a strategy to increase 7 th -graders’ average attendance from 78 
percent to 90 percent. To measure outcomes, the team collected 
data on the 2011—2012 attendance rates for 7 th graders. The data 
indicated that the 7 th graders had an average attendance rate of 
85 percent during the 2011—2012 school year. This is a seven 
percent improvement over the prior year, but it does not quite reach the team’s strategic target. 

The team records the findings on the Evaluating Progress and Improvement tool (6B, page 46) and decides to 
continue the strategy for another year, to try to make further progress. When they are available in the fall, team 
members will revisit the 2011—2012 NeSA scores to determine whether there has been any progress in improving 
student achievement. 


Student ID 

Attendance Rate 

714597 

65% 

527890 

79% 

980957 

66% 

926985 

75% 

993067 

70% 

101617 

59% 

747337 

92% 

979696 

65% 

257661 

88% 

230769 

92% 

905777 

83% 

910519 

91% 

864729 

95% 

353042 

89% 

762871 

100% 

720508 

93% 

385537 

95% 

121224 

100% 

829757 

95% 

144084 

96% 

623058 

100% 


62 


6B. Evaluating Progress and Improvement 


Focus Area: Improve students’ achievement, particularly as measured by the NeSA test. 

Key Challenge: Students with poor attendance seem to be achieving at a lower rate than students with good 
attendance. 

Strategic Statement: By the end of this school year, improve 7 th -graders’ overall average attendance from 78% 
to 90% by sending home weekly attendance reports that require a parent’s signature whenever a student has an 
unexcused absence. 

Date 

Baseline 

Measurement 

New 

Measurement 

Difference 

Has Desired Change Been 
Achieved? 

Next Steps 

Key Challenge 

Focus Area 

6/5/12 

78% 

85% 

+7% 

No 

Unknown 

• Continue strategy during 
2012-2013 school year. 

• Evaluate progress in 
attendance rates again 
in December, 2012 and 
at the end of the school 

year. 

• Collect NeSA scores 
in the fall and evaluate 
whether progress was 
made in the focus area. 

December 

2012 




Y N 

Y N 


June 

2013 




Y N 

Y N 
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Organizational Tools 


At the end of their cycle of inquiry, the data team reviewed the items that had been recorded on the Parking Lot Tool 
(7A, page 47). These questions and data findings would form the basis of their next cycle of data inquiry. 

7A. Parking Lot Tool 


Questions to Return to Later 

Is there a correlation between math scores and reading 
scores? 

Which teachers are producing the best outcomes and 
what are they doing differently in their classrooms? 

Which grades show the most improvement in NeSA and 
why? 

Look at the correlation between attendance rates and 
NeSA outcomes for grade levels besides 7 th grade. 


Data Findings to Return to Later 

Cross has a majority male population in her classroom. 

Overall, almost half of the 7 ,h -grade scored below 
proficiency on the NeSA-M. 

There are two students with 100% attendance who 
exceeded expectations in both NeSA-M and NeSA-R. 

• Both were in Grimes’ class. 

• What are they, or Grimes, doing differently? 


The team also reviewed the Data Cycle Overview (7B, page 47) to clarify and celebrate the completed work. With the 
overview available, the team can return to the data cycle at any time in the future to review its decisions. 

7B. Data Cycle Overview 


Focus Area 

Improve student achievement, particularly as measured by the NeSA assessment. 

Focus Question 

What effects, if any, do attendance rates have on NeSA test outcomes for 7 ,h -grade students? 

Key Challenge 

Students with poor attendance seem to be achieving at a lower rate than students with good 
attendance. 

Top Driving Factor 

Parents are not aware that their children are skipping school. 

Strategy 

Send home weekly attendance reports that require a parent’s signature. 

Strategic 

Statement 
(including Target) 

By the end of this school year, improve 7 th -graders’ overall average attendance from 78% to 
90% by sending home weekly attendance reports that require a parent’s signature whenever a 
student has an unexcused absence. 
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Appendices 



Appendix A: Glossary 


Term 

Definition 

Achievement 

Data 

Data that indicate how students are performing on tests of academic knowledge or skill. 

Area of Concern 

Factors that may impact the challenge but that the team cannot directly influence or change. 

Area of Control 

Factors that the team can directly change. 

Area of influence 

Factors that the team cannot directly change but that the team can possibly manipulate to 
move toward change. 

Causal 

Questions 

Questions that aim to determine whether one or more variables causes or effects one or more 
other variables. 

Cycle of Data 
Inquiry 

The specific project during which the Data Team completes all of the Stages of the Data 

Inquiry process. Each question for inquiry leads to a new cycle of data inquiry. 

Data Inquiry 

A process of identifying questions, collecting and examining data, interpreting data, and 
proposing actions based on data. 

Data Source 

The data that are under study or analysis. 

Demographic 

Data 

Data that provide information on characteristics of a population or sample. Common 
demographic variables include gender, race/ethnicity, socioeconomic status, and geographic 
location. 

Descriptive 

Questions 

Questions that aim to describe or summarize what is going on or what currently exists. These 
questions do not lead to predictions, associations between variables, or causal statements. 

Dichotomous 

Questions 

Questions that have only two possible answers. Common examples are those questions that 
lead to answers of “yes or no,” “up or down,” or “male or female.” 

Disaggregated 

Data 

Data that have been broken down according to subgroup identifiers to enable analysis of the 
data set according to subgroups. 

Interim 

Assessments 

Assessments that are administered during instruction to evaluate students’ knowledge and 
skills relative to a specific set of academic goals. 

MAP 

Northwest Evaluation Association's Measures of Academic Progress (MAP) assessment, a 
computer adaptive assessment used by several Nebraska schools to meet the Nebraska 
Department of Education's norm-reference requirements. 

NeSA 

The Nebraska State Accountability test is a system of criterion-referenced tests in reading, 
mathematics, science, and writing. Reading (NeSA-R) and mathematics (NeSA-M) are 
administered to students in grades 3-8 and 1 1 ; science is administered to students in grades 

5, 8, and 1 1 ; and writing is administered to students in grades 8 and 1 1 . 

Outcomes Data 

Data that indicate how students fare in all areas but academic achievement at the end of a 
period of time, such as a semester, year, or K-12 education. 

Perception Data 

Data that provide information about respondents’ values, opinions, views, and beliefs, often 
collected through surveys or questionnaires. 

Program Data 

Data that describe the structures and processes in place within the system or program under 
inquiry. 

Relational 

Questions 

Questions that analyze the relationship(s) between two or more variables. 

Variable 

A characteristic or observation that differs among members of a sample or population. 
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Appendix B: Question Bank 


This question bank includes some common question formats and example questions. As data teams begin working, 
they may find it useful to pull from this bank in order to save time as they learn the data inquiry process. For all data 
sources, demographic and other sources may also be required to disaggregate the data. 

This is not an exhaustive list of possible questions and should only be used as inspiration or for use by a new data 
team struggling with question development. 

Most questions should have associated timeframes which designate the period(s) (e.g., in semesters or years) across 
which data will be collected. In general, if a static measure of performance is desired, data from the most recent time 
period should be acquired. If a trending measure is desired, data from a series of consecutive time periods should be 
acquired. 


Question Format 

Example 

Data Sources Required for the Example 


1. What is the correlation between (variable) and 

variable) [in subgroup]? 

What is the correlation between benchmark math 
tests and NeSA-M scores? 

Benchmark scores that directly precede the 

NeSA-M scores 

How are attendance rates related to NeSA scores for 
5 th graders? 

NeSA scores, for 5 th grade only 

Attendance rates, for 5 th grade only 


2. How are (sub group) performing in (content area) as measured by (test) over (time period)? 

How are 4 th -grade girls performing in reading as 
measured by DIBELS over the last three years? 

DIBELS scores from the last three years for 4 th -grade girls 
only 

How have Title 1 students performed in reading 
over the course of this school year, as measured by 
interim course grades? 

Course grades from this school year for Title 1 students only 


3. How has (program or intervention) affected (data source)? 

How has our intensive reading program affected 
NeSA-R scores? 

NeSA-R scores from years during which program was 
implemented and at least one year prior 

How has the introduction of the Boys Town 

Education Model™ affected behavior rates at the 
high-school level? 

Behavior rates from the high school from years during which 
the program was implemented and at least one year prior 


4. What are the trends in (data source) over (time period)? 

What are the trends in graduation rates over the last 

1 0 years? 

Graduation rates for the last 10 years 

What are the trends in parent perception of the 
importance of education since we began parent 
nights four years ago? 

Parent perception data from the last four years 
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Question Format 

Example 

Data Sources Required for the Example 


5. What is (sub group)’s perception of (variable)? How does that correlate with (variable)? 

What are 5 th -grade students’ perceptions of their 
reading skills? How does that correlate with NeSA 
scores? 

Reading skills perception survey data from the 5 th -grade 
NeSA-R scores for the 5 th grade 

What are the middle school students’ perceptions 
of school culture? How does that correlate with 
classroom grades? 

Middle school perception survey data related to school 
culture 

Middle school classroom grades 


6. How does (data source) compare across (subgroups)? 

How do MAP math scores compare across 
classrooms? 

MAP math scores 

Disaggregated by classroom 

How do NeSA-M scores compare across student 
ethnic groups? 

NeSA-M scores 

Disaggregated by ethnicity 


7. What was the impact of (program) on (outcome)? 

What was the impact of additional reading on 
student NeSA-R achievement? 

NeSA-R scores from both before and after the program 
implementation 

What was the impact of using study guides on unit 
test scores? 

Unit test scores from both before and after the program 
implementation 


8. What is the relationship between (test) scores and (variable)? 

What is the relationship between ACT scores and 

ACT scores 

graduation rates? 

Graduation rates 

What is the relationship between MAP reading 

MAP reading scores 

scores and retention rates? 

Retention rates 
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Appendix C: Color-Coding the Data 


One possible approach to identifying patterns in the data is color-coding. Bright colors can be used to highlight all of 
the places where the data show students scoring below a certain threshold. The stoplight method suggests using the 
three colors of the stoplight to highlight the lowest third (red), the middle third (yellow), and the highest third (green) 
in a data set. For example, students below proficiency may be red, at proficiency may be yellow, and above proficiency 
may be green (Learning Point Associates, 2004). 

Below is an example of this method. The table consists of classroom data that show the raw score and proficiency 
level of 1 5 students on a single assessment. This is a comprehensive data set— a user can consider proficiency levels or 
raw scores at the classroom level and disaggregated by student, free or reduced-lunch status, and gender. The example 
has been color-coded according to proficiency and sorted by gender. 


Student ID 

Gender 

FRL 

Score 

Proficiency Level 

23456 

F 

NA 

14 

Exceeds 

67890 

F 

NA 

4 

Below 

13451 

F 

Free 

15 

Exceeds 

24562 

F 

Reduced 

10 

Exceeds 

46784 

F 

NA 

11 

Exceeds 

57895 

F 

Free 

4 

Below 

70987 

F 

NA 

3 

Below 

12345 

M 

Free 

15 

Exceeds 

34567 

M 

Reduced 

8 

Meets 

45678 

M 

NA 

13 

Exceeds 

56789 

M 

Reduced 

2 

Below 

35673 

M 

Free 

9 

Meets 

68906 

M 

Free 

9 

Meets 

80987 

M 

Reduced 

9 

Meets 

98765 

M 

Reduced 

8 

Meets 
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Appendix D: Creating Visual Displays of the Data 


There are many common types of graphs that can be easily created in Word or Excel. The type of graph that is 
chosen will depend largely on the data being shared. 

Line Graphs 

These graphs are best used to track data changes over a period of time. The following graph displays the 4 th -graders’ 
average math test scores over the first 1 0 weeks of school. 

Line graphs can also be used to compare two groups over the same period of time. In the second example, the 
data have been disaggregated by gender in order to show the average math scores for girls and boys in the 4 th grade. 
Having both lines on the same graph allows for comparison of these two subgroups over the first 10 weeks of school. 


4 ,h -Grade Students’ Average Math Test Scores 

100% r 


80% U 


60% 


40% U 


20% P 


0% 1 

Week Week Week Week Week Week Week Week Week Week 
1 234567 89 10 
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4 ,h -Grade Students’ Average Math Test Scores by Gender 


100 % 


80% 


60% 


40% 


20 % 


0% 



Week Week Week Week Week Week Week Week Week 
1 234567 89 


Boys 

Girls 


Bar Graphs 

These graphs are best used to compare variables across different groups. The following example displays the average 
math scores across the grade levels in ABC School. 


Average Math Scores in ABC School: Fall 2011 



Bar graphs can also be used to compare two groups on a single measure. In the next example, the data from the 
previous example have been disaggregated by gender in order to show the average math scores for girls and boys in 
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the school. Having both sets of data on the same graph is helpful in seeing how certain grade levels or genders are 
performing as compared to others. 


Average Math Scores In ABC School: Fall 2011, By Gender 



Grade 1 Grade 2 Grade 3 Grade 4 Grade 5 Grade 6 


■ Boys 

■ Girls 


Pie Charts 

These graphs are best used to compare parts of a whole. They are most often used to show and quickly compare 
percentages and do not show changes over time. The example below displays responses to a school climate survey. 


ABC School is a Good Place to Go to School 
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The pie chart shows that more than half of the respondents stated they “Agree” or “Strongly Agree” with the 
statement that “ABC School is a good place to go to school.” 

Scatterplots 

These graphs are best used to identify relationships between two variables. The variable that is considered explanatory 
typically is placed on the horizontal axis and the response variable is placed on the y axis. They are most useful when 
there are a large number of data points. Scatterplots are not recommended for use with very few data points. 

In the example below, the MAP percentiles for 62 4 th -grade students at ABC School are plotted on a scatterplot. 

4 ,h -Grade Attendance Rates and MAP Test Percentiles at ABC School 



10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 


Attendance Rate 


The attendance rate is considered explanatory, because the hypothesis is that attendance rates are affecting MAP 
percentile scores. 

The dots on the graph indicate a positive correlation between attendance and MAP percentiles (as attendance rates 
increase, MAP percentiles increase). A line can be drawn on the graph to emphasize this trend. The dots that fall well 
outside the trend line are outliers— they may have other factors causing them to fall outside the general trend seen with 
the majority of the population. 
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